Intermodal synchronization of surgical data

ABSTRACT

Systems and methods are provided in which local tissue diagnostic measurements are correlated with archival local tissue diagnostic data from prior tissue analyses to supplement diagnostic measurements with tissue analysis data from prior tissue analyses having similar local tissue diagnostic data. The tissue analysis data may include information such as pathology data, outcome data, and diagnosis data. The archived local tissue diagnostic data and the tissue analysis data may be stored in a database, and employed for a wide variety of methods, involving preoperative, intraoperative, and/or postoperative phases of a medical procedure. Methods and systems are also provided for displaying, on a medical image shown in a user interface, hyperlinked reference markers associated with tissue analyses, where the reference markers are shown at locations corresponding to local tissue analyses, and where associated diagnostic data and/or tissue analysis may be viewed by selecting a given reference marker.

CROSS-REFERENCE TO RELATED APPLICATIONS

This document is a continuation application claiming the benefit of, andpriority to, the following documents: U.S. patent application Ser. No.15/597,830; titled “INTERMODAL SYNCHRONIZATION OF SURGICAL DATA,” andfiled on May 17, 2017; U.S. patent application Ser. No. 14/855,054 filedSep. 15, 2015; International PCT Patent Application No.PCT/CA2014/050269 filed on Mar. 14, 2014; U.S. Provisional ApplicationNo. 61/801,282, titled “SYSTEMS AND METHODS FOR PATHOLOGY TRACKING” andfiled on Mar. 15, 2013; U.S. Provisional Application No. 61/800,911,titled “HYPERSPECTRAL IMAGING DEVICE” and filed on Mar. 15, 2013; U.S.Provisional Application No. 61/801,746, titled “INSERT IMAGING DEVICE”and filed on Mar. 15, 2013; U.S. Provisional Application No. 61/818,255,titled “INSERT IMAGING DEVICE” and filed on May 1, 2013, U.S.Provisional Application No. 61/800,155, titled “PLANNING, NAVIGATION ANDSIMULATION SYSTEMS AND METHODS FOR MINIMALLY INVASIVE THERAPY” and filedon Mar. 15, 2013; U.S. Provisional Application No. 61/924,993, titled“PLANNING, NAVIGATION AND SIMULATION SYSTEMS AND METHODS FOR MINIMALLYINVASIVE THERAPY” and filed on Jan. 8, 2014; U.S. ProvisionalApplication No. 61/798,867, titled “SYSTEM AND METHOD FOR RECORDING THETIME COURSE OF TOOLS THROUGH A PROCEDURE” and filed on Mar. 15, 2013,all of which are hereby incorporated herein by reference in theirentirety.

FIELD

The present disclosure relates to image guided medical procedures. Thepresent disclosure also relates to medical procedures involving tissueexcision, identification and/or pathology analysis.

BACKGROUND

Imaging and imaging guidance is becoming a more significant component ofsurgical care, from diagnosis of disease, monitoring of the disease,planning of the surgical approach, guidance during the procedure andfollow-up after the procedure is complete, or as part of a multi-facetedtreatment approach.

In many medical procedures, tissue samples are excised or examined, forexample, during the surgical removal of a tumor. Currently, in thefields of medical imaging and surgical diagnostics, taking a tissuesample and performing histopathology examination of the tissue sample,using a microscope, often with staining of that tissue, remains the goldstandard for tissue diagnosis. This histopathology examination involvesresection in a surgical suite and transfer of the sample to a pathologylaboratory.

However, this histopathology examination approach is fraught withproblems and issues. For example, current methods of tissue analysis areunable to accurately and painlessly access tissue and can result in thepossibility of seeding tumor cells through the biopsy process. Thishistopathology examination approach is also typically impractical toperform multiple biopsies to enable proper examination of heterogeneoustumors.

Tissue samples are also often mislabeling during the process, which canresult due to sample mix-up or labelling errors resulting in faultydiagnosis. Furthermore, pathology results may be discordant with theimaging results. Current workflow also often has a poor feedback loop toradiologists, hindering them from improving their diagnostic accuracyfor future cases. This also can result in an unnecessary delay betweenbiopsy and pathology results, resulting in a reduction in positivepatient outcomes.

SUMMARY

Systems and methods are provided in which local tissue diagnosticmeasurements are correlated with archival local tissue diagnostic datafrom prior tissue analyses to supplement diagnostic measurements withtissue analysis data from prior tissue analyses having similar localtissue diagnostic data. The tissue analysis data may include informationsuch as pathology data, outcome data, and diagnostic data. The archivedlocal tissue diagnostic data and the tissue analysis data may be storedin a database, and employed for a wide variety of methods, involvingpreoperative, intraoperative, and/or postoperative phases of a medicalprocedure. Methods and systems are also provided for displaying, on amedical image shown in a user interface, hyperlinked reference markersassociated with tissue analyses, where the reference markers are shownat locations corresponding to local tissue analyses, and whereassociated diagnostic data and/or tissue analysis may be viewed byselecting a given reference marker.

Accordingly, in one aspect, a computer implemented method of correlatinga local tissue diagnostic measurement with archival tissue analysis datais provided, the method comprising: obtaining local tissue diagnosticdata associated with one or more local tissue diagnostic measurementsperformed on a subject; accessing archival local tissue diagnostic dataand tissue analysis data associated with one or more prior local tissueanalyses; comparing, according to pre-selected similarity criteria, thelocal tissue diagnostic data associated with the one or more localtissue diagnostic measurements and the archival local tissue diagnosticdata associated with the one or more prior local tissue analyses;identifying one or more similar prior local tissue analyses havingarchival local tissue diagnostic data satisfying the pre-selectedsimilarity criteria; and providing tissue analysis data associated withthe one or more similar prior local tissue analyses.

In another aspect, a system for correlating a local tissue diagnosticmeasurement with archival tissue analysis data is provided, the systemcomprising: a control and processing system comprising one or moreprocessors and memory coupled to said one or more processors, saidmemory storing instructions, which, when executed by said one or moreprocessors, causes said one or more processors to perform operationscomprising: obtaining local tissue diagnostic data associated with oneor more local tissue diagnostic measurements performed on a subject;accessing archival local tissue diagnostic data and tissue analysis dataassociated with one or more prior local tissue analyses; comparing,according to pre-selected similarity criteria, the local tissuediagnostic data associated with the one or more local tissue diagnosticmeasurements and the archival local tissue diagnostic data associatedwith the one or more prior local tissue analyses; identifying one ormore similar prior local tissue analyses having archival local tissuediagnostic data satisfying the pre-selected similarity criteria; andproviding tissue analysis data associated with the one or more similarprior local tissue analyses.

In another aspect, a computer implemented method of displaying tissueanalysis information on a user interface is provided, the methodcomprising: obtaining a medical image of at least a portion of a subjectand displaying the medical image on the user interface; obtaining localtissue information corresponding to one or more local tissue analysesperformed on the subject; obtaining location data identifying a locationcorresponding to each local tissue analysis, wherein the location datais spatially registered to the medical image; displaying one or morereference markers in the medical image, wherein: each reference markeris associated with one of the local tissue analyses; and each referencemarker is displayed, in the medical image, at the location at which itsassociated local tissue analysis was performed; receiving input from anoperator identifying a selected reference marker associated with aselected local tissue analysis, thereby identifying selected localtissue information; and presenting at least a portion of the selectedlocal tissue information associated with the selected local tissueanalysis.

In another aspect, a system for displaying tissue analysis informationon a user interface is provided, the system comprising: a control andprocessing system interfaced with a display device, said control andprocessing system comprising one or more processors and memory coupledto said one or more processors, said memory storing instructions, which,when executed by said one or more processors, causes said one or moreprocessors to perform operations comprising: obtaining a medical imageof at least a portion of a subject and displaying the medical image onthe user interface; obtaining local tissue information corresponding toone or more local tissue analyses performed on the subject; obtaininglocation data identifying a location corresponding to each local tissueanalysis, wherein the location data is spatially registered to themedical image; displaying one or more reference markers in the medicalimage, wherein: each reference marker is associated with one of thelocal tissue analyses; and each reference marker is displayed, in themedical image, at the location at which its associated local tissueanalysis was performed; receiving input from an operator identifying aselected reference marker associated with a selected local tissueanalysis, thereby identifying selected local tissue information; andpresenting at least a portion of the selected local tissue informationassociated with the selected local tissue analysis.

In another aspect, a computer implemented method of correlatingpreoperative tissue analysis data with archival tissue analysis datafrom one or more prior medical procedures is provided, the methodcomprising: obtaining preoperative tissue analysis data associated witha subject; accessing archival tissue analysis data associated with oneor more prior medical procedures; accessing time-dependent medicalprocedure data recorded during the one or more prior medical procedures;comparing, according to pre-selected similarity criteria, thepreoperative tissue analysis data and the archival tissue analysis dataassociated with the one or more prior medical procedures; identifyingone or more similar prior medical procedures having archival tissueanalysis data satisfying the pre-selected similarity criteria; andprocessing the time-dependent medical procedure data associated with theone or more similar prior medical procedures to replay at least aportion of the medical procedure.

In another aspect, a system for correlating preoperative tissue analysisdata with archival tissue analysis data from one or more prior medicalprocedures is provided, the system comprising: a control and processingsystem comprising one or more processors and memory coupled to said oneor more processors, said memory storing instructions, which, whenexecuted by said one or more processors, causes said one or moreprocessors to perform operations comprising: obtaining preoperativetissue analysis data associated with a subject; accessing archivaltissue analysis data associated with one or more prior medicalprocedures; accessing time-dependent medical procedure data recordedduring the one or more prior medical procedures; comparing, according topre-selected similarity criteria, the preoperative tissue analysis dataand the archival tissue analysis data associated with the one or moreprior medical procedures; identifying one or more similar prior medicalprocedures having archival tissue analysis data satisfying thepre-selected similarity criteria; and processing the time-dependentmedical procedure data associated with the one or more similar priormedical procedures to replay at least a portion of the medicalprocedure.

In another aspect, a computer implemented method of suggesting one ormore steps of a surgical plan based on archival surgical plan data fromone or more prior medical procedures is provided, the method comprising:obtaining tissue analysis data associated with a subject; accessingarchival tissue analysis data associated with one or more prior medicalprocedures; accessing surgical plan data associated with the one or moreprior medical procedures; comparing, according to pre-selectedsimilarity criteria, the tissue identification data and the archivaltissue analysis data associated with the one or more prior medicalprocedures; identifying one or more similar prior medical procedureshaving archival tissue analysis data satisfying the pre-selectedsimilarity criteria; and processing the surgical plan data associatedwith the one or more similar prior medical procedures to generate one ormore steps of a suggested surgical plan; and communicating the one ormore steps of the suggested surgical plan.

In another aspect, a system for suggesting one or more steps of asurgical plan based on archival surgical plan data from one or moreprior medical procedures is provided, the system comprising: a controland processing system comprising one or more processors and memorycoupled to said one or more processors, said memory storinginstructions, which, when executed by said one or more processors,causes said one or more processors to perform operations comprising:obtaining tissue analysis data associated with a subject; accessingarchival tissue analysis data associated with one or more prior medicalprocedures; accessing surgical plan data associated with the one or moreprior medical procedures; comparing, according to pre-selectedsimilarity criteria, the tissue identification data and the archivaltissue analysis data associated with the one or more prior medicalprocedures; identifying one or more similar prior medical procedureshaving archival tissue analysis data satisfying the pre-selectedsimilarity criteria; and processing the surgical plan data associatedwith the one or more similar prior medical procedures to generate one ormore steps of a suggested surgical plan; and communicating the one ormore steps of the suggested surgical plan.

In another aspect, a computer implemented method of correlating apreoperative surgical plan with archival surgical plan data from one ormore prior medical procedures is provided, the method comprising:obtaining preoperative surgical plan data associated with a medicalprocedure to be performed on a subject; accessing outcome dataassociated with one or more prior medical procedures; accessing archivalsurgical plan data employed during the one or more prior medicalprocedures; comparing, according to pre-selected similarity criteria,the preoperative surgical plan data and the archival surgical plan dataassociated with the one or more prior medical procedures; identifyingone or more similar prior medical procedures having archival surgicalplan data satisfying the pre-selected similarity criteria; and providingoutcome data associated with the one or more similar prior medicalprocedures.

In another aspect, a system for correlating a preoperative surgical planwith archival surgical plan data from one or more prior medicalprocedures is provided, the system comprising: a control and processingsystem comprising one or more processors and memory coupled to said oneor more processors, said memory storing instructions, which, whenexecuted by said one or more processors, causes said one or moreprocessors to perform operations comprising: obtaining preoperativesurgical plan data associated with a medical procedure to be performedon a subject; accessing outcome data associated with one or more priormedical procedures; accessing archival surgical plan data employedduring the one or more prior medical procedures; comparing, according topre-selected similarity criteria, the preoperative surgical plan dataand the archival surgical plan data associated with the one or moreprior medical procedures; identifying one or more similar prior medicalprocedures having archival surgical plan data satisfying thepre-selected similarity criteria; and providing outcome data associatedwith the one or more similar prior medical procedures.

In another aspect, method of performing tissue analyses while performinga tissue resection procedure on a subject is provided, the methodcomprising: during a tissue resection procedure, performing, with aspatially tracked local tissue analysis device, a plurality of localtissue analyses, such that different local tissue analyses correspond todifferent tissue locations that are exposed during the resectionprocedure; employing tracking data associated with the spatially trackedlocal tissue analysis device to determine, in a reference framespatially registered to a medical image of the subject, the locationdata corresponding to each local tissue analysis; recording local tissuediagnostic data from each local tissue analysis in association with itscorresponding location data; constructing a spatial profile of theresected tissue by correlating the local tissue diagnostic data with thelocation data.

In another aspect, a method of performing intraoperative tissue analysiswhile performing a tissue resection procedure on a subject is provided,the method comprising: during a tissue resection procedure: performing alocal ex-vivo diagnostic measurement on a biopsy sample of a tumor,thereby obtaining a reference measurement associated with tumor tissue;and during subsequent tissue resection: intermittently performing localin-vivo diagnostic measurements on exposed tissue; and comparing thereference measurement to each in-vivo measurement to identify thepresence or absence of tumor tissue in the exposed tissue.

A further understanding of the functional and advantageous aspects ofthe disclosure can be realized by reference to the following detaileddescription and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments will now be described, by way of example only, withreference to the drawings, in which:

FIG. 1 is a diagram illustrating an exemplary navigation system tosupport minimally invasive access port-based surgery.

FIG. 2 is a diagram illustrating various components of system forperforming image-guided port based medical procedures.

FIG. 3 is a diagram illustrating a human brain into which an access porthas been inserted, establishing an open conduit for providing access totissue within the brain.

FIG. 4A is a flow chart illustrating the processing steps involved in aport-based surgical procedure using a navigation system.

FIG. 4B is a flow chart illustrating the processing steps involvedregistering a patient to an intraoperative reference frame.

FIG. 5 is a diagram illustrating an example implementation of computercontrol system for implementing the various methods disclosed herein.

FIG. 6A is a diagram illustrating an axial view of the brain in which atumor is present.

FIGS. 6B-6E are diagrams, together, illustrating an example userinterface, in which a regional image has reference markers shown at thelocations corresponding to tissue analyses.

FIG. 7 is a flow chart illustrating an example method of displayinglocation-specific and hyperlinked tissue analysis information in a userinterface.

FIGS. 8A and 8B are diagrams, together, illustrating the selectabledisplay, in a user interface, of tissue analysis information identifiedby searching a tissue analysis database.

FIG. 9 is a diagram illustrating an example embodiment involving fouraspects of patient care.

FIG. 10 is a flow chart illustrating an example method of identifyingsimilar prior tissue analyses by performing a similarity analysisbetween local diagnostic data and archival local tissue diagnostic datastored in a tissue analysis database.

FIG. 11 is a flow chart illustrating a method of selecting suitablediagnostic modalities for use during a medical procedure.

FIG. 12 is a flow chart illustrating an example method of determiningsimilarity among different tissue regions based on spectroscopicmeasurements.

FIG. 13 is flow chart illustrating an example method of obtainingoutcome data associated with a prior medical procedures having a similarsurgical plan to a proposed medical procedure.

FIGS. 14A and 14B are diagrams, together, illustrating example searchalgorithms that may be employed to search archival data sets.

FIG. 15 is a diagram illustrating an example embodiment involvingspecific utilization of regional imaging, point imaging, and pathologydata to link imaging and pathology results in a single patient, andlinking results across subjects.

FIG. 16 is a diagram illustrating an example embodiment involvingspecific utilization of preoperative imaging, pathology, and pointsource imaging data to facilitate decision making for treatment andsurgical planning

FIG. 17 is a diagram illustrating an example embodiment involvingspecific utilization of preoperative imaging to facilitate decisionmaking for tissue differentiation and treatment.

FIGS. 18A and 18B are diagrams, together, illustrating views of tissue,from the perspective of looking through an access port.

FIG. 18C is a diagram illustrating a probe interrogating an island oftissue through an access port.

FIG. 19 is a diagram illustrating a manner in which multiple tissuemetrics are utilized to characterize tissue of interest.

FIG. 20 is a diagram illustrating an example embodiment involvingspecific utilization of postoperative imaging in the context of expectedoutcomes.

DETAILED DESCRIPTION

Various embodiments and aspects of the disclosure will be described withreference to details discussed below. The following description anddrawings are illustrative of the disclosure and are not to be construedas limiting the disclosure. Numerous specific details are described toprovide a thorough understanding of various embodiments of the presentdisclosure. However, in certain instances, well-known or conventionaldetails are not described in order to provide a concise discussion ofembodiments of the present disclosure.

As used herein, the terms, “comprises” and “comprising” are to beconstrued as being inclusive and open ended, and not exclusive.Specifically, when used in the specification and claims, the terms,“comprises” and “comprising” and variations thereof mean the specifiedfeatures, steps or components are included. These terms are not to beinterpreted to exclude the presence of other features, steps orcomponents.

As used herein, the term “exemplary” means “serving as an example,instance, or illustration,” and should not be construed as preferred oradvantageous over other configurations disclosed herein.

As used herein, the terms “about” and “approximately” are meant to covervariations that may exist in the upper and lower limits of the ranges ofvalues, such as variations in properties, parameters, and dimensions. Inone non-limiting example, the terms “about” and “approximately” meanplus or minus 10 percent or less.

Unless defined otherwise, all technical and scientific terms used hereinare intended to have the same meaning as commonly understood to one ofordinary skill in the art. Unless otherwise indicated, such as throughcontext, as used herein, the following terms are intended to have thefollowing meanings:

As used herein, the phrase “medical instrument” refers to a tool,instrument, or other implement employed during a medical procedure. Amedical instrument may be provided in various forms, such as, but notlimited to, a handheld or robotically positioned tool, or a componentthat is attached to, or inserted into, a subject during a surgical ormedical procedure. Non-limiting examples of medical instruments include,but are not limited to, scalpels, bi-polar devices, suction devices,cutting devices, clamping devices, access ports, and forceps.

As used herein, the phrase “operator” refers to a user, medicalpractitioner, surgeon, imaging technician, or other individual or groupof individuals involved in operating medical instruments, devices andequipment during a medical procedure.

As used herein, the phrase “tracking system” refers to a systemconfigured to track the position and/or orientation of one or moreobjects, such as locations of a subject and/or surgical instruments. Insome embodiments, the tracking system may be configured to track theposition and/or orientation of an imaging device (such as an opticalcamera). A tracking system may also be employed to track the positionand/or orientation of an access port or other component that is attachedto, or inserted into, a subject or subject. In one example, a trackingsystem may employ a pair of infrared cameras to track the position andorientation of active or passive infrared spheres (fiducials) attachedto one or more objects, such as the Polaris® system from NDI.

As used herein, the phrase “navigation system” refers to a system thatprocesses and spatially registers preoperative image data to anintraoperative reference frame, and displays the position andorientation of one or more tracked items relative to the preoperativeimage data. A navigation system may interface with, or include, atracking system, in order to track the items. In some exampleimplementations, hardware associated with the navigation system mayinclude a computer system, a display, and a tracking system.

As used herein, the phrase “phase of the medical procedure” refers to agiven step, or set of sequential steps, within a medical procedure. Inanother example, a phase of a medical procedure need not be a given stepor set of sequential steps in a procedure, but may relate to the use ofa specific tool or set of tools within a given step of a medicalprocedure.

As used herein, the phrase “intraoperative” refers to an action,process, method, event or step that occurs or is carried out during atleast a portion of a medical procedure. Intraoperative, as definedherein, is not limited to surgical procedures, and may refer to othertypes of medical procedures, such as diagnostic and therapeuticprocedures.

As used herein, the phrase “access port” refers to a cannula, conduit,sheath, port, tube, or other structure that may be inserted into asubject in order to provide access to internal tissue, organs, or otherbiological substances. In some embodiments an access port may directlyexpose internal tissue, for example via an opening or aperture at adistal end thereof, and/or via an opening or aperture at an intermediatelocation along a length thereof. In other embodiments an access port mayprovide indirect access, via one or more surfaces that are transparent,or partially transparent, to one or more forms of energy or radiation,such as, but not limited to, electromagnetic waves and acoustic waves.

As used herein, the phrase “local tissue analysis” refers to an actiontaken to, or event associated with, the local analysis of tissue during,and optionally after, a medical procedure. In one example, a localtissue analysis may involve obtaining a biopsy sample during a medicalprocedure, and performing an analysis on the biopsy sample eitherintraoperatively or postoperatively. In another example, a local tissueanalysis may involve obtaining a diagnostic measurement of a localregion (e.g. a subset of a region associated with a medical image, or asubset of an anatomic region within a subject). Understood is that alocal tissue analysis involving a diagnostic measurement may beperformed to obtain one or more spot or point measurements (optionallycombining a plurality of local spot or point measurements to constructan image) or an image of a local tissue region.

As used herein, the phrase “tissue analysis data” refers to dataobtained after having performed a local tissue analysis. For example, inthe case in which a local tissue analysis is performed as a biopsy withpostoperative analysis, the tissue analysis data may be measurements(e.g. cell morphology, cell type, microscopy images, etc.) obtained whenperforming the analysis of the biopsy sample. In the case in which alocal tissue analysis is performed to obtain a local diagnostic image,the tissue analysis data may include the local image data. Non-limitingexamples of local diagnostic images include, for example, a white lightimage, a hyperspectral image, a polarization-sensitive image, an opticalcoherence tomography image, an ultrasound image, and a magneticresonance imaging image. In another non-limiting example, in the casewhen a local tissue analysis if performed to obtain a spot or pointmeasurement within a region of interest, the local tissue analysis datamay be a spectrum, such as a Raman spectrum or an optical spectrum.

As used herein, the phrase “local”, when used in association with adiagnostic measurement, refers to a diagnostic measurement obtained ator near a tissue of region of interest. For example, a local diagnosticmeasurement may be made with a local diagnostic non-imaging device, suchas a Raman probe, or with a local diagnostic imaging device, such as anexoscope or magnetic resonance imaging probe. A local diagnosticmeasurement has a location associated therewith, where the location maybe shown in a regional or global image of a subject. The phrase“regional”, when used in association with a diagnostic image, refers toan image including both a tissue region of interest, and othersurrounding tissue structure.

As used herein, the phrase “diagnosis data” refers to data orinformation associated with the diagnosis of a medical condition, suchas a type of tumor or a stage of a tumor. Diagnosis data may be basedon, or include, pathology data.

As used herein, the phrase “pathology data” refers to informationassociated with pathology testing of a tissue sample. Pathology data mayinclude a pathology report. In another non-limiting example, pathologydata may include information associated with one or more pathologicaltissue types identified from the local tissue analysis such as, but notlimited to, tumor type, tumor stage, tumor size, and tumor cellinformation.

As used herein, the phrase “subject” refers to human or non-humansubjects or patients.

Some example embodiments of the present disclosure provide methods andsystems that involve the integration of imaging and tissue analysis. Insome example embodiments a combination of regional, and local imaging,and tissue biopsy or local analysis, may be employed to inform decisionmaking and treatment selection during or after a medical procedure. Someexample embodiments described below provide systems and methods forintegrating and updating preoperative and intraoperative plans based onprior medical procedures having, for example, similar local tissueanalysis data, similar pathology data, and/or similar surgical plans.

In some example embodiments described below, systems and methods areprovided in which three-dimensional positions (between or withinsubjects) associated with local tissue analyses (e.g. biopsy or in-vivomeasurements) are associated with preoperative or intraoperative images,and/or with information associated with prior tissue analyses, such asprior outcomes (e.g. subject outcomes and/or economic outcomes),archival tissue analysis, and/or pathology data (which may be stored inan electronic data base including subject information). Furthermore, insome embodiments information recorded during previous medical proceduresmay be employed to assist with the performing or planning of a medicalprocedure.

While many of the examples and illustrations provided in the presentdisclosure relate to minimally invasive neurological procedures, such asprocedures involve resection of brain tumors, understood is that thescope of the present disclosure is intended to include and be applicableto, a wide range of medical procedures as further described below.

Example Minimally Invasive System for Performing Image-Guided MedicalProcedure

FIGS. 1 and 2 illustrate an example automated system for performingvarious embodiments of the present disclosure, providing a non-limitingexample pertaining to a computer-assisted minimally-invasiveneurological surgical procedure employing an access port. FIG. 1illustrates a perspective view of a minimally invasive port basedsurgical procedure. Surgeon 101 conducts a minimally invasive accessport-based surgery on a subject 102 in an operating room (OR)environment. An automated system including an equipment tower, cameras,displays, and tracked instruments assists surgeon 101 during the medicalprocedure. One or more operators 103 may also present to operate,control and provide assistance for the one or more aspects of thesystem.

FIG. 2 illustrates various example components of an automated system forassisting a medical procedure involving an access port. The systemincludes one or more imaging devices (for example, volumetric, wholeorgan, regional, point, or tool based), surgical guidance devices,software systems, databases, tissue specimen handling devices, andtracked medical instruments (e.g. surgical tools) as an integratedsystem. As described in various example embodiments below, the systemmay be configured to correlate three-dimensional positions on or withinsubjects with pathology samples, preoperative or intraoperative images(volumetric, regional, point or tool based), and patient and economicoutcomes, and an electronic data base of patient information.

The example automated system includes an automated robotic arm 105,which supports an optical video scope 110 (and associated illumination),video display 115 for displaying a video image from optical video scope110, navigation display 116 for providing a navigation user interface, atracking device 120 for tracking various medical instruments within thesurgical field, and a control and processing unit 400 for controllingvarious devices (such as the robotic arm 105) and providing surgicalnavigation. A secondary display may provide output of the trackingdevice 120. The output may be shown in axial, sagittal and coronal viewsas part of a multi-view display.

The example embodiment shown illustrates control and processing system400 as residing in an equipment tower in a single tower configuration,connected to dual displays 115 and 116. However, understood is thatother configurations may alternatively be employed (for example, a dualtower configuration and/or a single display configuration). Furthermore,an equipment tower may also be configured with a UPS (universal powersupply) to provide for emergency power, in addition to a regular ACadapter power supply.

As described in detail below, in some embodiments, control andprocessing system 400 may include, or may be interfaced with, one ormore recording devices or software modules that provide real-timerecording of one or more aspects of the medical procedure. For example,the system may be configured to capture one or more of audio, video,sensory and multi-modal (e.g. CT, MR, US, etc.) inputs from differentsources. All relevant data may be received via one or more recordingdevices (for example, stored in the equipment tower) and stored inmemory by a recording module. The one or more aspects of the medicalprocedure may be automatically recorded from the outset of theprocedure, or may be controlled by an operator and/or administrator. Inother embodiments, the procedure may be automatically recorded (bydefault), but there may be an option to override or delete the recordingduring the medical procedure or after the medical procedure has beencompleted.

Referring again to FIG. 2, a subject's head is held in place by a headholder 125, and inserted into the head is an access port 130 andintroducer 135 (having fiducial markers attached thereto). Introducer135 is shown received within access port 130 in the figure, and istracked using tracking system 120. A guide clamp 133 for holding accessport 130 may be provided. Guide clamp 133 can optionally engage anddisengage with access port 130 without needing to remove the access portfrom the subject. In some embodiments, access port 130 can slide up anddown within clamp 133 while in the closed position. A locking mechanismmay be attached to or integrated with guide clamp 133, and canoptionally be actuated with one hand, as described further below.

Articulated arm 134 may be provided with an attachment point to holdguide clamp 133. Articulated arm 134 may have up to six degrees offreedom to position guide clamp 133. Articulated arm 134 may be attachedor attachable to a point based on subject head holder 125, or anothersuitable subject support, to ensure when locked in place, guide clamp133 cannot move relative to the subject's head. The interface betweenguide clamp 133 and articulated arm 134 may be flexible, or optionallylocked into place. Flexibility is desired so the access port can bemoved into various positions within the brain, but still rotate about afixed point.

An example of such a linkage that can achieve this function is a slenderbar or rod. When access port 130 is moved to various positions, the baror rod will oppose such a bend, and move the access port 130 back to thecentered position. Furthermore, an optional collar may be attached tothe linkage between the articulated arm, and the access port guide, suchthat when engaged, the linkage becomes rigid. Currently, no suchmechanisms exist to enable positioning access port 130 in such a manner

The position of the subject may be initially determined and/orcontinuously tracked intraoperatively by tracking system 120. A set ofpreoperative images associated with the anatomy of interest of thesubject may be obtained prior to surgery. These images may beintraoperatively registered to the subject, for example, by way ofsurface matching, sets of known touch points (e.g., tip of nose, temple,and ears) and/or fiduciary markings that can be identified on thesubject and in the associated images. These points or surfaces areregistered to the tracking coordinate frame through a definedregistration process. Once registered, medical instruments, and theassociated subject images can be tracked in real-time, and shown invarious manners on a computer monitor.

The example automated system illustrated in FIG. 2 is configured for theapplication of minimally invasive brain surgery, using an access port toprovide a conduit within the head, allowing access to internal braintissue for surgical, therapeutic, or diagnostic applications. The figureshows an intracranial access port which may be employed in neurologicalprocedures in order to provide access to internal tissue pathologies,such as tumors. One example of an intracranial access port is theBrainPath™ surgical access port provided by NICO, which may be insertedinto the brain via an obturator (introducer) with an atraumatic tip.Such an access port may be employed during a surgical procedure, byinserting the access port via the obturator that is received within theaccess port to access an internal surgical site.

FIG. 3 illustrates the use of an access port, showing a human brain 140into which an access port 130 has been inserted, thereby establishing anopen conduit providing access to tissue deep within the brain. Surgicalinstruments may then be inserted within the lumen of the access port inorder to perform surgical, diagnostic or therapeutic procedures, such asresecting tumors as necessary. This approach allows a surgeon, orrobotic surgical system, to perform a surgical procedure involving tumorresection in which the residual tumor remaining after is minimized,while also minimizing the trauma to the intact white and grey matter ofthe brain. In such procedures, trauma may occur, for example, due tocontact with the access port, stress to the brain matter, unintentionalimpact with surgical devices, and/or accidental resection of healthytissue. For example, access port based procedures may be employed forother surgical interventions for other anatomical regions such as, butnot limited to, spine, knee, and any other region of the body that willbenefit from the use of an access port or small orifice to access theinterior of the human body.

Referring again to FIG. 2, in order to introduce the access port 130into the brain, introducer 135 with an atraumatic tip may be positionedwithin the access port and employed to position the access portionwithin the head. As noted above, introducer 135 (or access port 130) mayinclude fiducials for tracking. These fiducials may be passive or activefiducials, such as reflective spheres for passive infrared detection viaan optical camera, or, for example, pick-up coils in the case of anelectromagnetic tracking system. The fiducials are detected by trackingsystem 120 and their respective positions are inferred by trackingsoftware (which may reside within tracking system 120, or may reside,for example, within control and processing unit 400).

Once access port 130 is inserted into the brain, introducer 135 may beremoved to allow for access to the tissue through the central opening ofaccess port 130. However, once introducer 135 is removed, access port130 can no longer be directly tracked in real time (according to theexample embodiment shown in FIG. 2 in which no fiducials are attached toaccess port 130). In order to track the position and orientation ofaccess port 130, it may be indirectly and intermittently tracked by apointer tool having fiducials that are detectable by tracking system120.

Although the example system described in FIGS. 1 and 2 relates to aneurosurgical procedure, understood is that the systems and methodsdescribed herein are not intended to be limited to neurosurgicalprocedures or port-based procedures, and may be employed for a widerange of medical procedures. Examples of other types of medicalprocedures including orthopedic, trauma, gastrological, cardiac,gynecological, abdominal, otolaryngology (or ENT—ear, nose, throatconditions), spinal, thoracic, oral and maxillofacial, urological,dental, and other surgical, diagnostic or therapeutic medicalprocedures. It is further noted that while many of the exampleembodiments described herein employ external imaging, such as imagingwith an external video scope, understood is that various internalimaging devices, such as endoscopic or catheter imaging devices, mayadditionally or alternatively be employed. Noted is that embodiments ofthe present disclosure may be employed within or adapted to proceduresemploying telesurgical or shared-control systems.

In many of the example embodiments described below, each medicalinstrument that is to be tracked may have a fiducial attached thereto(e.g. passive or active fiducial markers, such as reflective spheres oractive LED lighting emitted from at least 3 points on a device) so thatthe position and orientation of the instrument can be determined. In oneexample implementation, the fiducial markers may be employed todetermine a reference position on medical instrument (such as a centralpoint), and an axis of the medical instrument (such as a longitudinalaxis of a tool).

Example Methods of Performing Access Port Based Medical Procedure

FIG. 4A is a flow chart illustrating the processing steps involved in anexample port-based surgical procedure using a navigation system. Thefirst step involves importing the port-based surgical plan (step 302). Adetailed description of the process to create and select a surgical planis outlined in the disclosure “PLANNING, NAVIGATION AND SIMULATIONSYSTEMS AND METHODS FOR MINIMALLY INVASIVE THERAPY,” a United StatesPatent Publication based on a United States Patent Application, whichclaims priority to U.S. Provisional Patent Application Ser. Nos.61/800,155 and 61/924,993, which are both hereby incorporated byreference in their entirety.

As outlined above, an example surgical plan may include preoperative 3Dimaging data (e.g., MRI, ultrasound, etc.) overlaid with inputs (e.g.,sulcal entry points, target locations, surgical outcome criteria, andadditional 3D image data information) and displaying one or moretrajectory paths based on the calculated score for a projected surgicalpath. Understood is that the present example embodiment is providedmerely as an illustrative example plan, and that other surgical plansand/or methods may also be employed without departing from the scope ofthe present disclosure.

Once the plan has been imported into the navigation system in step 302,the subject is affixed into position using a head or body holdingmechanism. The head position is also confirmed with the subject planusing the navigation software, as shown in step 304.

Registration of the subject is then initiated in step 306. The phrase“registration” or “image registration” refers to the process oftransforming sets of data into a common coordinate system. Registereddata may be provided in the form of multiple images, data from differentsensors, times, depths, or viewpoints. The process of registration isemployed in the present application for medical imaging in which imagesfrom different imaging modalities are co-registered.

Appreciated is that there are numerous registration techniques availableand one or more of them may be employed according to the embodiments ofthe present disclosure. Non-limiting examples of registration methodsinclude intensity-based methods which compare intensity patterns inimages via correlation metrics, while feature-based methods findcorrespondence between image features such as points, lines, andcontours. Image registration algorithms may also be classified accordingto the transformation models they use to relate the target image spaceto the reference image space. Another classification can be made betweensingle-modality and multi-modality methods. Single-modality methodstypically register images in the same modality acquired by the samescanner/sensor type, for example, a series of CT images can beco-registered, while multi-modality registration methods are used toregister images acquired by different scanner/sensor types or pulsesequences, for example in MRI and PET. Multi-modality registrationmethods are often used in medical imaging of the head/brain, as imagesof a subject are frequently obtained from different scanners. Examplesinclude registration of brain CT/MRI images or PET/CT images for tumorlocalization, registration of contrast-enhanced CT images againstnon-contrast-enhanced CT images, and registration of ultrasound and CT.

FIG. 4B is a flow chart illustrating the further processing stepsinvolved in registration as outlined in FIG. 4A. In one exampleimplementation, the method may employ fiducial touchpoints as shown at340. In such a case, the process involves first identifying fiducials onimages at step 342, then touching the touchpoints with a trackedinstrument (step 344). Next, the navigation system computes theregistration to reference markers (step 346).

In another example implementation, registration can be performed byconducting a surface scan procedure, as shown at 350. The first stepinvolves scanning a portion of the body (e.g., the face) using a 3Dscanner (step 352). The face surface is then extracted from the MR/CTdata (step 354). Finally, surfaces are matched to determine registrationdatapoints. Upon completion of either the fiducial touchpoint 340 orsurface scan 350 procedures, the data extracted is computed and used toconfirm registration, as shown in step 308.

In another example implementation, recovery of loss of registration maybe provided. For example, during a medical procedure, a handheld medicalinstrument may be tracked using a tracking system, and a representationof the instrument's position and orientation may be provided anddisplayed as an overlay on a previously acquired or current image (suchas a three-dimensional scan) of a subject's anatomy obtained with animaging device or system (such as ultrasound, CT or MRI).

To achieve such an image overlay, a registration is needed between thecoordinate frame of a tracking system, the physical location of thesubject in space, and the coordinate frame of the corresponding image ofthe subject. This registration is typically obtained relative to atracked reference marker, which is placed in a fixed position relativeto the patient anatomy of interest and thus can be used as a fixedreference for the anatomy. Generally this can be accomplished byattaching the reference to a patient immobilization frame (such as aclamp for skull fixation in neurosurgery), which itself is rigidlyattached to the subject (for example, as shown in FIG. 2).

However, the reference may be held to the frame, for example, through anarm, which can be bumped and accidentally moved, which creates a loss ofregistration. Additionally, since the reference marker must bepositioned so that it is visible by the navigation hardware (typicallyrequiring line-of-sight for optical tracking, or otherwise within theobservation or communication field of the tracking system), this tendsto position the reference such that it is in the open thus moresusceptible to accidental interaction and loss of registration. Insituations of lost registration, a surgical procedure tends to bestopped while a new registration is computed, although this may notalways be possible if, for example, the registration fiducial points orpatient skin surface are no longer accessible due to the progression ofthe surgical procedure, and thus creating a need for a fullre-registration or, in some cases even disabling navigation for theremainder of the procedure.

Referring again to FIG. 4A, once registration is confirmed in step 308,the subject is draped (step 310). Typically, draping involves coveringthe subject and surrounding areas with a sterile barrier to create andmaintain a sterile field during the surgical procedure. The purpose ofdraping is to eliminate the passage of microorganisms betweennon-sterile and sterile areas. Upon completion of draping (step 310),the patient engagement points are confirmed at step 312, and craniotomyis then prepared and planned (step 314).

Upon completion of the prep and planning of the craniotomy (step 312),the craniotomy is then cut, where a bone flap is temporarily removedfrom the skull to access the brain (step 316). Registration data isupdated with the navigation system at this point (step 322). Theengagement within craniotomy and the motion range is then confirmed instep 318. Once this data is confirmed, the procedure advances to thenext step of cutting the dura at the engagement points and identifyingthe sulcus (step 320). Registration data is also updated with thenavigation system at this point (step 322).

In one example embodiment, by focusing the camera's gaze on the surgicalarea of interest, this registration update can be manipulated to ensurethe best match for that region, while ignoring any non-uniform tissuedeformation affecting areas outside of the surgical field (of interest).Additionally, by matching overlay representations of tissue with anactual view of the tissue of interest, the particular tissuerepresentation can be matched to the video image, and thus tending toensure registration of the tissue of interest.

For example, in one example implementation, a video image may beprovided in which a post-craniotomy real-time intraoperative opticalimage of the brain surface (i.e. exposed brain) is provided with anoverlay of the preoperative (and registered) sulcal map, and theregistration may be corrected by providing input manipulating aligningthe preoperative sulcal map with the sulcal profile that is observablein the real-time intraoperative image.

In another example implementation, a video image may be provided inwhich a post-craniotomy real-time intraoperative optical image of theexposed vessels on the brain surface (i.e. exposed brain) is providedwith an overlay of preoperative image of vessels (obtained via imagesegmentation of preoperative image data, co-registered withintraoperative position), and the registration may be corrected byproviding input manipulating aligning the preoperative vessels with theexposed vessels that are observable in the real-time intraoperativeimage.

In another example implementation, a video image may be provided inwhich a post-craniotomy real-time intraoperative optical image of anexposed tumor (i.e. exposed brain) is provided with an overlay of apreoperative (and registered) image of the tumor (obtained via imagesegmentation of preoperative image data), and the registration may becorrected by providing input manipulating aligning the preoperativetumor image with the exposed tumor that are observable in the real-timeintraoperative image.

In another example implementation, a video image may be provided inwhich a real-time intraoperative optical image of a nasal cavity isprovided with an overlay of a preoperative (and registered) image ofbone rendering of the bone surface (obtained via image segmentation ofpreoperative image data), and the registration may be corrected byproviding input manipulating aligning the preoperative bone image withthe bone surface that is observable in the real-time intraoperativeimage.

In other embodiments, multiple cameras can be used and overlaid withtracked instrument(s) views, and thus allowing multiple views of thedata and overlays to be presented at the same time, which can provideeven greater confidence in a registration, or correction in more thandimensions/views.

Thereafter, the cannulation process is initiated, as shown at step 324.Cannulation involves inserting a port into the brain, typically along asulcal path as identified in step 320, along a trajectory plan.Cannulation is an iterative process that involves repeating the steps ofaligning the port on engagement and setting the planned trajectory (step332) and then cannulating to the target depth (step 334) until thecomplete trajectory plan is executed (step 324).

The surgeon then performs resection (step 326) to remove part of thebrain and/or tumor of interest. The surgeon then decannulates (step 328)by removing the port and any tracking instruments from the brain.Finally, the surgeon closes the dura and completes the craniotomy (step330).

Example Implementation of Control and Processing Unit

Referring now to FIG. 5, a block diagram of an example systemconfiguration is shown. The example system includes control andprocessing unit 400 and a number of external components, shown below.

As shown in the FIG. 5, in one embodiment, control and processing unit400 may include one or more processors 402, a memory 404, a system bus406, one or more input/output interfaces 408, and a communicationsinterface 410, and storage device 412. Storage device 412 may beemployed to store information associated with a medical procedure, suchas, but not limited to, local tissue analysis data, surgical plan data,pathology data, and recorded time-dependent medical procedure data.

Control and processing unit 400 is interfaced with other externaldevices, such as tracking system 120, data storage 442, and externaluser input and output devices 444, which may include, for example, oneor more of a display, keyboard, mouse, foot pedal, microphone andspeaker. Data storage 442 may be any suitable data storage device, suchas a local or remote computing device (e.g. a computer, hard drive,digital media device, or server) having a database stored thereon.

In the example shown in FIG. 5, data storage device 442 may includearchival information associated with prior tissue analyses, and controland processing unit 400 may be programmed to process such information toperform one or more of the methods described below. As shown in theexample implementation illustrated in FIG. 8, data storage device 442may include the following examples of information associated with aprior tissue analyses: diagnosis data 450, outcome data 451, archivallocal tissue analysis data 452, recorded time-dependent medicalprocedure data 454, planning data 456 (e.g. a surgical plan having beenfollowed during a medical procedure including a local tissue analysis),and additional information associated with subjects associated with theprior tissue analyses, such as, but not limited to, demographic,prognostic, prior history, and/or genetic information. Although datastorage device 442 is shown as a single device in FIG. 5, understood isthat in other embodiments, data storage device 442 may be provided asmultiple storage devices.

Medical instruments 460, such as a tissue resection device (for example,the Myriad tissue resection device manufactured by NICO), a biopsydevice, or a local diagnostic measurement device (e.g. point-based orimaging based), are identifiable by control and processing unit 400.Medical instruments 460 may be connected to, and controlled by, controland processing unit 400, or may be operated or otherwise employedindependent of control and processing unit 400. Tracking system 120 maybe employed to track one or more of medical instruments 460 andspatially register the one or more tracked medical instruments to anintraoperative reference frame.

Control and processing unit 400 is also interfaced with a number ofconfigurable devices that may be tracked by tracking system. Examples ofdevices 420, as shown in the figure, include one or more imaging devices422, one or more illumination devices 424, robotic arm 105, one or moreprojection devices 428, and one or more displays 115. The one or moreimaging devices 422 may include one or more local diagnostic measurementdevices (e.g. point-based or imaging based). Tracking system 120 may beemployed to track one or more of devices 422 and spatially register themto an intraoperative reference frame.

Embodiments of the disclosure can be implemented via processor(s) 402and/or memory 404. For example, the functionalities described herein canbe partially implemented via hardware logic in processor 402 andpartially using the instructions stored in memory 404, as one or moreprocessing engines 470. Example processing engines include, but are notlimited to, user interface engine 472, tracking engine 474, motorcontroller 476, image processing engine 478, image registration engine480, procedure planning engine 482, navigation engine 484.

As described in detail below, one or more processing engines may beprovided for process information associated with prior tissue analyses,and such engines are represented by analysis engine 486. For example, insome embodiments, an analysis engine is provided to evaluate similaritycriteria between of one or more local tissue analyses performed on asubject, and prior local tissue analyses stored in data storage device442, or otherwise accessible, such as through an external network.Examples of such methods are described in the forthcoming descriptionand accompanying flow charts. As described in detail below, similaritycriteria may involve the evaluation of one or more metrics associatedwith one or more local tissue analyses performed on a subject, and oneor more prior local tissue analyses, where the prior local tissueanalyses may be associated with the medical history of the subjectand/or a collection of other subjects.

In some example embodiments, the processing engines may be employed toperform methods including, but not limited to, tracking 3D position andorientation data for the purpose of spatially registering diagnosticdevices capable of performing local diagnostic measurements (e.g. pointbased or imaging measurements); tracking locations of biopsy specimensto maintain 3D position and imaging information; recording biopsysampling locations relative to the timing of the biopsy; recordingsurgical tool, and imaging device positions and actuation throughout amedical procedure; determining and recording margin boundaries in atissue of interest in a virtual manner; locating regions on a 3D imageand correlating pathology information to such regions; andcharacterizing tissue based on one or more tissue metrics, and employingsuch metrics to search a database including prior tissue analysis data,and ranking results based on a variable weighted metric based algorithm.

Understood is that the system is not intended to be limited to thecomponents shown in the FIG. 5. One or more components control andprocessing 400 may be provided as an external component or device. Inone alternative embodiment, navigation module 484 may be provided as anexternal navigation system that is integrated with control andprocessing unit 400.

Some embodiments may be implemented using processor 402 withoutadditional instructions stored in memory 404. Some embodiments may beimplemented using the instructions stored in memory 404 for execution byone or more general purpose microprocessors. Thus, the disclosure is notlimited to a specific configuration of hardware and/or software.

While some embodiments can be implemented in fully functioning computersand computer systems, various embodiments are capable of beingdistributed as a computing product in a variety of forms and are capableof being applied regardless of the particular type of machine orcomputer readable media used to actually effect the distribution.

At least some aspects disclosed can be embodied, at least in part, insoftware. That is, the techniques may be carried out in a computersystem or other data processing system in response to its processor,such as a microprocessor, executing sequences of instructions containedin a memory, such as ROM, volatile RAM, non-volatile memory, cache or aremote storage device.

A computer readable storage medium can be used to store software anddata which when executed by a data processing system causes the systemto perform various methods. The executable software and data may bestored in various places including for example ROM, volatile RAM,nonvolatile memory and/or cache. Portions of this software and/or datamay be stored in any one of these storage devices.

Recording Time-Dependent Information Associated with a Medical Procedure

In some embodiments, the one or more events during a medical proceduremay be temporally and spatially tracked, and this information can belogged. For example, if a tissue specimen is captured, the time at whichis it captured can be recorded. This time can be correlated withlocation through the use of tracked tools in a navigation system(tracked using an optical approach, or an EM-based tracking system). Thelocation information will correspond to intraoperative MRI or CT (or PETor SPECT). The pathology information (microscope images, local imaging)can be associated with the time-stamp to allow that information to bestored in an appropriate database. This database can be searchable bypatient (to see how the same tissue looked under different modalities)or by disease type (to see how the same disease (validated by pathology)looks under different modalities), or by modality (to see what diseasesare possible given a certain result (i.e. what possible tissues couldhave a given Raman spectra).

By tracking the movements and locations of all tools throughout theprocedure, it is also possible to determine efficiencies in theoperation—for example that the surgeon uses a particular tool for a veryshort or very long time, or that certain steps are less efficient thanthey could be. This information can be used by the surgeons and hospitaladministrators to properly estimate surgical times for optimumscheduling.

As noted in FIG. 5, one or more medical instruments 460 and devices 420may be tracked using tracking system 120. Such instruments and/ordevices may be tracked, for example, using fiducials markers. Fiducialmarkers may be passive or active fiducials, such as reflective spheresfor passive infrared detection via an optical camera, or, for example,pick-up coils in the case of an electromagnetic tracking system. Thefiducials are detected by tracking system 120 and their respectivepositions are inferred by tracking software (which may reside withintracking system 120, or may reside, for example, within control andprocessing unit 400. Such tracking allows the position and orientationof the instruments and/or devices to be determined and recorded. In oneexample implementation, the fiducial markers may be employed todetermine a reference position on medical instrument or device, (such asa central point), and an axis of the medical instrument or device (suchas a longitudinal axis of a tool).

In some embodiments, tracked position and orientation data associatedwith one or more instruments and/or devices is recorded during a medicalprocedure. For example, in one example embodiment, the time-dependentposition and orientation, and optionally other state information orconfiguration parameters associated one or more instruments or devicesis recorded. For example, information that may be recorded may include,but is not limited to, tip offset, tip deflection, the state of controlbuttons and status, accuracy of measurement, visibility to a trackingsystem, the identity of a tool that is being used as a registrationreference, and/or a registration reference transform may be recordedand/or processed.

In one example implementation, any or all of the recorded informationmay be recorded, along with time data (e.g. transmitted to a recordingdevice), at sequential time points, at particular time points, or onlyat time points for which the information to be recorded has changedrelative to its previously recorded value.

In some example embodiments, a timestamp is associated with one or more,or all, of recorded information. Each timestamp may be matched to aglobal timestamp associated with other events recorded during themedical procedure, such as a global timestamp associated with videocaptures (such as during a surgery, or sub-procedures such as when acauterizer was on, when cutting, when tissue pathology samples wereobtained, or other procedures). Further data can be integrated from anysource from which a timestamp can be matched, including audio streams,video streams, state information of any imaging displays (e.g. data set,hanging protocol, window level, position, orientation, etc.) or otherevents of interest (e.g. registration of biopsy samples, containers,results, etc.).

In one example embodiment, the time-dependent position and orientationof a medical instrument or device may be recorded as a vector relativeto a fixed reference position (such as a patient fixed reference tool ora tracking camera centered on a coordinated frame). An orientation canbe recorded, for example, as a quaternion or axes vectors, anddeflection can be recorded as an offset, or another parameterizedgeometric representation (such as spline parameters).

In various embodiments, several parameters may be tracked and recordedat each time for an instrument, depending on the data of relevance andany assumptions made (or variables held constant).

In one example implementation, seven parameters may be recorded torepresent the time-dependent position and orientation of a device orinstrument. The six parameters are x, y, z, roll, pitch and yaw. The7^(th) parameter being the time instance at which specific orientationinformation is recorded. In other example implementations, fewerparameters may be used, if one or more assumptions are made (if, forexample, there is no desire to track or record a direction of rotation,such as an elongated instrument rotating around a longitudinal axis).More parameters can be used if the data is represented in otherconvenient ways, such as a 4×4 (16 parameter) transformation matrix inhomogeneous coordinates, or a quaternion representation of rotations asan example. In other embodiments, fewer or more parameters may betracked, as required or desired by the configuration for use.

The representation, transmission, receiving, recording, storing, andprocessing of information pertaining to a medical procedure as describedherein may allow for efficient storage, communication, and processing ofthe state of tools and instruments involved in a procedure. For example,by storing such data as discrete data points pertaining to specificpoints in time, less storage space may be required than, for example, avideo capture of a screen display of a navigation system and the storageof this information may enable other features and functions, asdescribed below.

As noted above, in some example embodiments, one or more othertime-stamped data streams, in addition or in alternative to a datastream recorded for a tracked instrument, may be recorded for subsequentplayback and/or analysis.

In an example embodiment, once at least a portion of the time course ofa procedure has been recorded as described herein, the spatial (positionand orientation) data points and associated timestamps may be processedfor subsequent analysis. For example, an analysis may be performed todetermine, based on the retrospective data, one or more metricsassociated with a procedure (such as surgical metrics). Examples of suchmetric include, but are not limited to, an amount of time that a tool orinstrument was used, an amount of time that a tool or instrument spentat each depth of insertion, an amount of time that a tool or instrumentspent near a certain feature (such as a vessel, nerve or lesion), and/oran extent of coverage of the various tools and instruments availableduring the procedure.

As noted above various aspects of a medical procedure may be recorded(e.g. in a database) in a seamless manner so that the surgeon or anoperator can utilize the volume of data for future care of the samesubject, other subjects, or training purposes. Data streams may include,but are not limited to, any or more of the following (or additional datastreams): video streams; audio streams; location and position of alltracked instruments and items; the state of any imaging displays (dataset, hanging protocol, window level, position, orientation, etc.; otherevents of interest (registration of biopsy samples, containers,results). This would enable the local or distributed playback, at somepoint in the future, the entire operative procedure in part or in whole.As noted above, one example implementation would be to allow the surgeonto “pause” the playback and then take control of the “paused” imagingdisplays and execute what-if scenarios for post-mortem analysis, audit,education, or training One would be able to assess how the surgicalprocedure would have proceeded if the approach was slightly differentapproach than was actually used during the recorded session. Theplayback could even proceed using a different display angle, differentdata sets, or other differences from the originally recorded procedure.

Alternatively, a paused playback point could be used as the startingpoint for a new simulation, where the current position and orientationsof tracked items and states of displays become the starting point forvirtual manipulation of both the instruments and the imaging displays.One could even record the progress of the simulation in the same way asa live procedure so that it can also be used for time shifted audit,analysis, and training Finally, several recorded simulations ofdifferent parts of one procedure could be made in different physicallocations by different people, can be reviewed and analyzed at anotherlocation by a different team, and then the “best” parts of eachsimulation could be stitched together to provide a final “best”solution.

In some embodiments, a common source of time stamps for samples isprovided at high enough resolution and low enough latency so thatrecording of samples of various stream can be effectively distributedduring real time, and then later amalgamated.

In some example embodiments the data points and timestamps may befurther processed to simulate intraoperative instrument/device motion.This simulation may be provided as a controllable replay. Unlike a videofeed that is only viewable from the angle of recording, the presentexample embodiment permits the recreation and playback, ofinstrument/device motion through other angles by way of the recordeddata points and timestamps. In other words, the viewpoint from which asimulation is rendered may be controllable by an operator.

Additionally, such an embodiment may be implemented to provide controlsduring playback, as if the instruments/devices were being manipulatedlive, which, when the system is associated with other imaging devicesand systems, and/or spatially registered data from imaging devices andsystems, permits slices and/or views of the image data to be visualizedand manipulated in ways that are different than the ones used during theprocedure. This embodiment may be employed to provide a more flexible,dynamic and/or controllable review of a procedure, rather than astraight static playback of a video recording of the procedure, which islimited to displaying only the image data that was recorded, frompre-set views, at the time of the procedure.

In one example implementation positions may be played back in real-timeand also, for example, sped up, slowed down, paused, and searched.Additionally, an operator may recall some or all activities performednear a particular location in a spatially-constrained search function.Such an embodiment provides a “location aware” search function. Forexample, a 3D brain structure may be reconstructed using preoperativemedical images such as MRI or CT. A user can then position a pointingdevice or a cursor at a specific anatomical position (e.g. theparacentral sulcus) and initiate a search. The search routine may querya database with previously recorded position and time information ofeach surgical tool and identify all instances where tip of a surgicaltool was present in a predefined (and configurable) vicinity of theidentified location. The location may be also identified using 3orthogonal planar projections of the preoperative medical image, such asthe multiplanar reconstruction image layout typically used by aradiologist. The position and associated time stamp information of thesurgical tools may be alternatively stored as a sequential datastructure. The query information (position information in this case) maybe then used to search the list after sorting the data by position usingany searching algorithm for sequential data.

In some embodiments, the positions and orientations of trackedintraoperative imaging devices and systems (such as, but not limited to,ultrasound probes, optical coherence tomography (OCT) probes,spectroscopy probes, intraoperative MRI coils, intraoperative CT, orfluoroscopy devices, and the like) can be correlated with the acquiredimages and analyzed in relation to other imaging data positioned andoriented to match the tools and instruments data points. If timestampsare available on the imaging devices/systems data, such timestamps canalso be matched to the timestamps associated with the data pointspertaining to the recorded position and orientation data for theinstrument(s), so as to provide time-synchronized images and/or videos.

For example a hand-held ultrasound probe may be used shortly aftercraniotomy during brain tumor resection to identify presence ofvasculature immediately below the dura. The position relative to thebrain where the ultrasound image (typically, a B-mode scan) is obtainedand time instance when the image is recorded may be captured. Thisrecording, when synchronized with the time component of the 7 parametertime-dependent position and orientation information recording describedpreviously, allows for exact association of intraoperative images withspecific time points in the surgical procedure.

Such synchronized recording requires synchronization of a referenceclock the time that is utilized by all recording systems in the room(OR). This can be achieved, for example, by configuring the recordingdevices to record the present time (e.g. a time referenced to UTC).Alternatively, if the recording devices are not recording standard time,they may be synchronized using a common server that provides timeinformation to multiple clients, where one client may be acomputer-based data acquisition systems used to record the position ofsurgical tools and another client may be a computer-based imageacquisition system used to acquire ultrasonic images. Another client maybe a computer-based system used to acquire intraoperative MRI.Similarly, other separate computer-based image and patient'sphysiological system monitoring systems may act as clients to a commontime server. The server may be contacted by the clients at regularintervals to update the internal clock of each client. Hence, any driftin individual clocks of the clients can be adjusted. The end result istime synchronized recording of surgical tools, intraoperativephysiological state of the patient (e.g. dissolved oxygen level) andintraoperative images.

This synchronized data may be valuable as a training tool, as well as toassess efficient use of surgical workflows and associatedinstrumentation. Such information associated with utilization of varioustools and techniques in a particular surgical procedure may be analysedby health institutions to identify cost associated with surgicalprocedures and substantiate the charges communicated to insurancecompanies and other payers.

The preceding example embodiments may be employed for a variety ofpurposes. For example, the recorded information may be employed forreview of a medical procedure, for a variety of purposes, including, notlimited to, clinical, educational, legal, self-assessment, performanceassessment, audit, retrospective determination of exact location ofpoints of interest for subsequent imaging and follow-up assessment,and/or in-situ review of previous step(s) of a procedure during theconduct of the procedure, among others.

Furthermore, the preceding embodiments may be modified to allow anoperator to “pause” the playback of a recorded procedure, and then takecontrol of the “paused” imaging displays, and perform what-if scenariosfor post-mortem analysis, audit, education, or training A pausedplayback point could thus be used as the starting point for a newsimulation, where the current position and orientations of tracked itemsand states of displays become the starting point for virtualmanipulation of both instruments and imaging displays. The progress ofsuch a simulation can also be recorded in the same way as a liveprocedure so that it can also be used, for example, for time shiftedaudit, analysis, and training

After having paused the playback of a recorded medical procedure, anoperator may provide input specifying another angle, or may provideinput such that the playback proceeds with different informationdisplayed than was actually used during the recorded session. Theplayback may proceeds using a different display angle, different datasets, or other differences from the original recorded session.

In one example embodiment, both recordings and/or simulated video orvisualization data from different parts of a medical procedure could beobtained or created in different physical locations by different peopleand in different sessions. These parts can be reviewed and analyzed atanother location by a different team and the “best” parts of eachsimulation, as identified by a human, stitched together to provide afinal “best solution” for future training purposes.

For example, in the case of brain tumor resection one surgeon mayperform a real craniotomy and then acquire images using an ultrasoundprobe to identify the presence of sulcal folds or vasculature beneaththe dura. The position of surgical tools and ultrasound scans will berecorded as the first data set, first comprised of the position ofsurgical tools at regular time instances along with the ultrasonic dataacquired at specific time instances. The time instances act as a commonparameter that help synchronize the position information of the surgicaltools with the ultrasonic data acquired during the same procedure.

While the first surgeon would have proceeded with the complete surgicalprocedure, a second surgeon may use the recorded first data set as asimulated surgical scenario. The second surgeon may pause the playbackof the same surgical procedure at a time point. For illustrationpurpose, this time point may be shortly after completion of craniotomy.The second surgeon may then choose another simulated dural opening or adifferent simulated sulcal opening based on the presence of high-riskvasculature immediately below the dura—as observed from previouslydescribed ultrasound image. This action will generate a second data set.The second data set will have a pointer at the start of the data seriesto indicate the exact time point in the first data set where second dataset begins. New data sets can be created so that that one can view acombination real surgical recordings and simulated surgical recordingsand chose to observe different approaches to the same initial diseasecondition. All such alternative data set corresponding to the sameinitial disease condition may be simultaneously stored and retrieved ondemand and alternative intervention methods compared by a human Ifmetrics are associated with alternative clinical approaches describedabove, surgical steps with highest metrics or scores may be combinedsequentially to arrive at an overall optimal procedure.

The information recorded according to the preceding embodiments may bestored in an information store or database of procedures, optionallyalong with other data such as data obtained from a local tissueanalysis, or from an imaging system, and can be used as anintraoperative surgical data source, pathology correlation, and/or forfuture surgery/procedure planning and training Several examples of suchmethods are described below.

Methods involving Correlation of Diagnostic Data for Tissue Analysis

The following example embodiments provide methods involving the use oftissue analysis to supplement preoperative, intraoperative, orpostoperative analysis and guidance. Such methods are provided toaddress a fundamental problem of surgery, namely the problem of needing,but not obtaining, sufficient information pertaining to tissueidentification.

For example, during a neurosurgical resection procedure, a surgeontypically needs to decide, at various stages during the procedure,whether or not an exposed tissue region should be excised. There areseveral diagnostic modalities that the surgeon can use to attempt tomake such a determination. However, no single intraoperative modalitycan generally give a definitive conclusion. Indeed, there are severaltypes of brain tumors that appear similar to each other on MRI, however,are different in pathology. Hence, MRI alone is not sufficient toidentify the tumor. Examining the cells under a microscope is the goldstandard in tumor identification, but it is not feasible to perform onevery excised piece of tissue and often cannot be performedintraoperatively. An approach to arrive at a definitive conclusion mayinvolve the interrogation of a tissue region with one modality, in orderto initially narrow down the possible tissue types. Then a secondmodality could then be used that would supply more information andfurther narrow the tree of possibilities. Eventually, it may be possibleto definitively identify the tissue. Such a method is therefore complexand uncertain in its effectiveness.

Traditionally, there has been little integration between diagnosticimaging, surgery, and pathology in existing solutions. Imaging offers anopportunity to seamlessly present information between the disciplines ofradiology (diagnosis), surgery (treatment), neurology (outcomes) andpathology (tissue analysis). Imaging, and pathology analysis targeting aspecific region of interest can be correlated on the scale of the tissuesample size that is resected. Procedure outcomes are often dependent onthe percentage of total diseased tissue that is removed, compared to theamount of healthy tissue that is accidentally resected.

For achieving a more accurate pathology sample to imaging correlation, amore accurate method of registering a volume of interest of atissue-sampling device, and delivering the tissue-sampling device to aregion of interest within a subject, may be employed, as describedbelow. Furthermore, in order to locate a smaller volume of interest in alarger surgically excised volume of tissue of interest, a system thatcan perform diagnostic measurements on the tissue in an ex-vivo manner,using the appropriate contrast mechanism, may be employed.

In some example embodiments, the ability to perform local tissueanalysis on the scale of the volume of tissue to be resected, and totrack the resulting local tissue analysis data relative to excisedtissue, and to register to a more regional image of the tissue, may beemployed to obtain a correlation between local, regional, full volume,and pathology results. This information can be tracked relative to theoutcome of the medical procedure, or progression of the disease locally,as tracked by imaging on subsequent imaging procedures.

Accordingly, in some embodiments, an integrated system is provided thatenables imaging on a full volume, regional, or local basis, in thecontext of a medical procedure, and provides the appropriate imagingwith the appropriate tissue contrast to provide for diagnostic, surgicalextent and outcome prediction. Such a system may be employed, forexample, when larger regions are resected for the purpose of morecertainly resecting a small target of interest inside a larger volume(for instance 10 cubic centimeter volume), where the system provides thecapability of performing imaging of the resected surgical specimen insuch a manner that the smaller region of interest can be analyzed at ahigher resolution for pathological analysis. These and other systems,and associated methods, are described in detail below.

The following example embodiments illustrate various aspects of thepresent disclosure, in the context of an access port based neurosurgicaltumor resection procedure. As noted above, understood is that suchexample embodiments are not intended to limit the scope of the presentdisclosure to neurological procedures, and understood is that thesystems and methods disclosed herein may be readily adapted to, andemployed for, various other types of medical procedures.

In some embodiments, a system is provided to enable tracking ofindividual locations within a patient's anatomy, provide local imagingdata at the location, provide external imaging data when tissue isremoved (hand-shake image). External imaging may be performed as avolumetric or a surface scan.

In one embodiment, a sample is transferred to pathology in a labeledcontainer. The labeled container is uniquely identified in the system,and the system can locate the sample to a specific location in theimaging volume, through tool tracking, and to a set of in-vivo andex-vivo imaging sets. Any larger scans of the tissue of interest can beused to target specific regions of interest.

The system may comprise i) a navigation system that registers a set ofvolumetric imaging scans, to a patient frame of reference, ii)intraoperative imaging in a targeted region of interest, iii) softwaresystem to register preoperative imaging, intraoperative imaging, andimaging of pathology samples, iv) a database to store relevantinformation, including, but not limited to, patient data, or ElectronicMedical Records (EMR), Picture Archiving and Communication System(PACS),Treatment Plans, Lab Testing Reports, Pathology Reports and Imaging,Patient Outcomes (reports and lab tests), v) software system to search,weight metrics, calculate similarity or metrics, and rank based on saidmetrics. vi) a software system to present said results in the context ofthe decision making process (diagnostic scan, surgery, pathologydiagnosis, outcome evaluation), vii) means of sorting and imaging biopsysamples.

The system may be employed to track and measure comparable tissue samplemetrics throughout the process of diagnostic imaging, biopsy, treatmentplanning, surgery, and follow-up imaging. Thus, the system may providecomparable case information for patient(s) with similar metrics. Thesecomparisons can better inform the clinical specialist of similarimaging, pathology, or outcomes for a given imaging, pathology oroutcome condition for a specific patient.

In some embodiments, the system may utilize patient data alreadyavailable, by way of current diagnostic imaging scans, lab results,patient information (EMR), to better inform surgery, pathology andoutcomes for the case in progress. The information associated with thecase in progress would likewise be recorded, tracked, and submitted tosame informatics systems (EMR, image databases, lab results,), in amanner that they will contribute additional information for the nextcase In this way, the system may acts as an adaptive decision makingsystem such that more patients treated, and the more often informationis entered into a system for a patient, the more powerful the ability ofthe system to present more data to the physician for more effectivedecision making.

Referring now to FIG. 6A, an axial view of a brain 530 is illustrated,with a tumor 532 originating in the ventricle of the brain, and growingto the surface of the brain. The tumor is shown as three differenttextures, representing three different types of tumor cells. This isrepresentative of tumors, which can be heterogeneous in their biology,and thus their appearance on imaging, pathology, and their response totreatments.

A tracked medical instrument 534 is shown relative to the tumor andpointing to a location in the tumor. When used in conjunction with atracking system for tracking the instrument tip (a tip tracking strategymay be employed for flexible instruments, e.g. greater than 1 mm flex atthe tip inside the tissue of interest, or the resolution of interest forthat procedure), the position of the tracked medical instrument 534 isknown relative to the local tissue region of interest.

In one example embodiment, if medical instrument 534 is a biopsysampling device, or includes a biopsy device, and the biopsy instrumentactuation is measured in coordination with the sample removal, and thesample is may be stored or tracked in a manner that it can be uniquelyidentified relative to this local tissue region. Tissue analysis, suchas pathology results, can then be recorded and displayed relative to thelocation for which the sample was retrieved.

In another example embodiment, if medical instrument 534 is (orincludes) a local diagnostic measurement device the local tissuediagnostic data obtained from a local diagnostic measurement may bestored or tracked in a manner that such that the data can be uniquelyidentified relative to this local tissue region. The local tissuediagnostic data, such as a local image or a Raman spectrum, can then berecorded and displayed relative to the local tissue region at which thediagnostic measurement was made.

In one embodiment, the location associated with one or more tissueanalyses (e.g. a biopsy or a local diagnostic measurement) may be shownon a regional medical image that includes the local tissue region. Inorder to show the location of a given tissue analysis, the location dataassociated with the tissue analysis is spatially registered to themedical image data. This may be performed using registration methods,such as obtaining a preoperative medical image and spatially registeringthe preoperative image data to an intraoperative reference frame towhich the location data associated with the tissue analysis isregistered.

Referring now to FIG. 6B, an example implementation of such anembodiment is illustrated, in which the location of three tissueanalyses are shown by reference markers 540, 541 and 542 are shown, in auser interface, overlaid on medical image data.

As shown in FIGS. 6C and 6D, reference markers 540, 541 and 542 mayadditionally serve as selectable graphical hyperlinks to informationassociated with the tissue analyses. For example, in FIG. 6C, theselection, via input from an operator (e.g. a mouse click or the touchof a finger or stylus on a touch screen display) results in the displayof a pathology report 550 associated with a tissue sample obtained fromlocation 540. In another example implementation, shown in FIG. 6D, theselection, via input from an operator, of reference marker 540, resultsin the display of local tissue diagnostic data 555, including an MRimage 556 and a Raman spectrum 557 that were locally measured.

For example, if in-vivo image data is acquired concurrently with biopsysamples then the in-vivo imaging can be presented in the same context,and location, as the reference marker for the pathology results. Ifthere are no corresponding pathology results, the in-vivo data may beprovided in place of pathology results, as shown in FIG. 6D. Examples oflocal imaging modalities include OCT, high frequency ultrasound,Spectroscopy, MRI, MR Spectroscopy, tissue conductivity, electromagneticimaging, etc.

It is noted that the present example provides method of reviewing localtissue diagnostic data and tissue analysis results in a manner that isvery different from conventional methods. For example, some existingpathology software systems associate diagnostic image data based onhyperlinked text in a pathology report. In other words, the access tolocal diagnostic image data, or other diagnostic data, is providedthrough the pathology report. In contrast, the present embodiment allowsfor the graphical navigation to relevant location-specific diagnostic ortissue analysis data through a user interface, via the selection of ahyperlinked reference marker (e.g. a label, icon, text, or tag), on aregional medical image, where the position of the reference markercorresponds to the location at which a biopsy sample was acquired or alocal tissue diagnostic measurement was performed.

In some embodiments, illustrating in FIGS. 6B-6D, there may be multiplelocations for which hyperlinked data is available. In such a case, theuser may select one or more of the reference markers in order to view,or otherwise obtain (e.g. print, download, or email) the relevant data.

In some example embodiments, a single location may have multiple formsof associated local data. For example, as shown in FIGS. 6C and 6D,reference marker 540 has three different forms of associated data,including a pathology report (a form of tissue analysis data) and twoforms of diagnostic data (a local MR image and a Raman spectrum). FIG.6E illustrates one example in which a menu 560 may be provided todisplay a list of selectable items to display.

FIG. 7 provides a flow chart illustrating the present exampleembodiment. At 570, a medical image is obtained of at least a portion ofa subject and displayed on user interface. The medical image may beobtained, for example, preoperatively, using fiducial markers thatenable subsequent registration to an intraoperative reference frameassociated with a medical procedure during which the tissue analysis isperformed. In another example, the medical image, and the local tissueinformation associated with at least one local tissue analyses, may beintraoperatively obtained during a common medical procedure.

Local tissue information, corresponding to one or more local tissueanalyses, is then obtained at 572. Location data identifying thelocation corresponding to each local tissue analysis is obtained at 574,where the location data is spatially registered to the medical image. At574, reference markers corresponding to each tissue analysis aredisplayed in the medical image, at the locations corresponding to theirspatially registered location data. Input is then received from anoperator at 576, the input identifying a selected marker, and therefore,a selected tissue analysis. At least a portion of the local tissueanalysis information associated with selected tissue analysis is thenpresented, or is otherwise provided, to the operator.

Understood is that the variations of the embodiments shown in FIGS.6B-6E may be performed without departing from the scope of the presentdisclosure. For example, the selected local tissue information may bedisplayed in a separate window of the user interface. In one exampleimplementation, at least a portion of the selected local tissueinformation associated with the selected reference marker is displayedintraoperatively during a medical procedure, and wherein at least onesaid one or more local tissue analyses are performed during the medicalprocedure.

In one example implementation, at least one local tissue analysis maypertain to a previously medical procedure performed on the subject. Thelocal tissue diagnostic data associated with at least one local tissueanalysis may include additional tissue analysis information associatedwith a previous local tissue analysis that was performed atapproximately the same location.

Understood is that a tracking system typically has a positional errorassociated therewith. For example, the will generally be an errorassociated with tip position, at the particular time when a biopsysample or a local diagnostic measurement is obtained, for instance, dueto registration error and imaging related distortions. The estimatedspatial error associated with the location corresponding to a giventissue analysis can be estimated, and displayed as a visualrepresentation associated with the corresponding reference marker. Forexample, the visual representation can be provided as a circle, sphere,error bars, or other suitable representation.

In additional embodiments, the method may be adapted to track specificlocations that are registered across various modalities and resolutionsenables the user to “drop points” virtually throughout the surgicalcavity as tissue is resected. This allows a virtual margin to becreated. These points that are defined in a local area, are linked to alocal imaging frame. The region of interest is located in larger volumeby tracking of the imaging device, and imaging is recorded in synchronywith that tracking. If imaging at a larger scale is performed withoutsignificant tissue deformation, an image with a larger field of view canbe defined. In such a way, the larger fields of view can link to theentire imaging field. If the larger field of view can be imaged usingcontrast common to the preoperative, or intraoperative imaging, thesepoints can be registered between the various clinical utilizations ofthe system.

FIG. 8A illustrates another embodiment in which information pertainingto one or more similar tissue analyses may additionally or alternativelydisplayed in response to the selection of a given reference marker.Methods for identifying similar tissue analyses are described in detailbelow. As shown in the figure at 585, various form of tissue informationpertaining to one or more similar tissue analyses may be presented via auser interface, such as, but not limited to pathology data associatedwith similar prior tissue analyses, outcome data associated with similarprior tissue analyses, and/or diagnosis data associated with similarprior tissue analyses. Furthermore, as shown in FIG. 8B, local tissuediagnostic data 590 associated with similar prior tissue analyses may bepresented to the operator. These example embodiments, and relatedmethods, are described in detail below.

As noted above, in some embodiments, tissue information pertaining toprior tissue analyses may be provided, based on a determination ofsimilarity. In some embodiments, the determination of similarity may bemade based on a comparison between local tissue diagnostic dataassociated with the subject, and archival local tissue diagnostic dataobtained from a tissue analysis database. Such embodiments, involvingthe use of a tissue analysis database, may be employed according to awide variety of methods, and in a wide variety of medical procedures,and stages of a given medical procedure. The information stored in thedatabase may be tracked, updated and utilized as an adaptive evaluationtool, to search for similar results (pathology, imaging and outcomes) inthe history of the patient, patients with similar imaging/clinicalpresentations, and/or database with all patients' information and theirmedical history.

An example illustration of different stages of a surgical procedure, andtheir association to one or more tissue analysis databases, is shown inFIG. 9. The figure shows an example embodiment involving four stages ofdecision-making, namely diagnostic evaluation 505, surgical planning510, intraoperative surgery, diagnosis or treatment 515, andpostoperative analysis 520. These stages are shown in their relation toeach other, and with regard to one or more tissue identificationdatabases 500, which can be searched during one or more stages of amedical procedure. In this example, four aspects of patient care, wherethe use of a database linking registered imaging, pathology and outcomescan be utilized to improve diagnosis, surgical planning, surgical tissuedifferentiation and treatment and postoperative decision-making

In the example workflow shown in FIG. 9, the diagnostic modalitieslisted, include, but are not limited to, a set of whole organ, regional,or local diagnostic modalities, which may include imaging modalitiessuch as, magnetic resonance Imaging (MRI), computerized tomography (CT),positron emission tomography (PET), SPECT, ultrasound (US), x-ray,optical (visible light or sections of full EM spectrum), opticalcoherence tomography (OCT), photo-acoustic (PA) or regional imagingmodalities. These modalities can be acquired and shown as 1D, 2D, 3D, or4D (3D+time), data sets, any may be registered to the patient in adimensional and positional accurate manner Biopsy methods include coreor endoscopic biopsy, surgical biopsy (large section), aspiration, orother methods of removing tissue of interest for further pathologyanalysis.

It is noted that the phrase “outcome”, as used herein, refers toquantifiable methods to measure mortality and morbidity of the subject.This includes, but is not limited to, measurement of actual patientfunction, including direct measures of tissue viability, or higher-levelfunction, as well as in-direct measurements, tests and observations. Anoutcome may also refer to the economic outcome of a procedure (in aspecific or broad sense), and may include the time for the procedure,the equipment and personal utilization, drug and disposable utilization,length of stay, and indications of complications and/or comorbidities.

In some example embodiments that are described in detail below, tissueanalysis may be performed by comparing local tissue diagnosticmeasurements (obtained with one or more diagnostic modalities) witharchived local tissue diagnostic data. The archived local tissuediagnostic data is associated with prior tissue analyses for whichtissue analysis data, such as outcomes, pathology data, and diagnoses,are available. The tissue analysis data is stored in a tissue analysisdatabase (or two or more databases) with the associated archived localtissue diagnostic data. The local tissue analysis data (pertaining to asubject) may be employed to search the tissue analysis database toidentify one or more similar prior tissue analyses, and the tissueanalysis data associated with the similar prior tissue analyses may beprovided to the surgeon, practitioner, or operator, or processed andemployed for various uses and applications, examples of which aredescribed further below.

For example, referring now to FIG. 10, a flow chart is provided thatillustrates and example method for correlating a local tissue diagnosticmeasurement with archival tissue analysis data. At step 600, localtissue diagnostic data is obtained, where the local tissue diagnosticdata is associated with one or more local tissue diagnostic measurementsperformed on a subject.

For example, as described below, the local tissue diagnostic data may belocal imaging data, such as a MR image obtained via an insertable MRprobe, or local non-imaging data, such as locally measured Ramanspectrum. Although this local tissue diagnostic data may not besufficient to perform tissue analysis, it may be correlated witharchival local tissue diagnostic data from prior tissue analyses fromthe same or other subjects, as described below. In cases in which thelocal tissue diagnostic data pertains to more local tissue diagnosticmeasurements made with more than one diagnostic modality, the locationat which each local tissue analysis is made may be recorded, optionallyalong with a time stamp, as above described. The location may beemployed to correlate local tissue diagnostic data obtained for a commontissue location, but with different diagnostic modalities.

At step 605, archival local tissue diagnostic data and tissue analysisdata associated with one or more prior local tissue analyses is accessedor otherwise obtained. As noted above, tissue analysis data may includeinformation including, but not limited to, one or more of pathologydata, outcome data, tissue identification data, and/or diagnosis data.The archival local tissue diagnostic data, and the associated tissueanalysis data, pertain to previous local tissue analyses, and may beprovided in a tissue analysis database, as explained further below. Atstep 610, the local tissue diagnostic data associated with the one ormore local tissue diagnostic measurements, and the archival local tissuediagnostic data associated with the one or more prior local tissueanalyses, are then compared, according to pre-selected similaritycriteria.

The local tissue diagnostic data pertaining to the subject may beemployed to search the tissue analysis database for similar prior tissueanalyses. Non-limited example of diagnostic modalities include MRI (T1,T2, DWI, ADC, FA, SWI, MRS), CT, Ultrasound, SPECT, PET, Ramanspectroscopy, OCT, histological staining and high resolution opticalimaging (microscopy and otherwise). For example, if the local tissuediagnostic data obtained for the subject includes a Raman spectrum, atissue analysis database may the searched to find archival local tissuediagnostic data that was also measured via Raman spectroscopy (where thearchived Raman spectra are stored correlated with tissue analysis data),and the measured Raman spectrum for the subject may be compared with thearchival Raman spectrum to find a prior tissue analysis having a similarRaman spectrum.

At step 615, one or more similar prior local tissue analyses havingarchival local tissue diagnostic data satisfying the pre-selectedsimilarity criteria are identified, thereby identifying a prior tissueanalysis that may be representative of the local tissue region of thesubject. The tissue analysis data associated with the one or moresimilar prior local tissue analyses may then be provided, displayed, orotherwise processed for further uses, as described below.

As described above, the tissue analysis database may be generated byperforming multiple tissue analyses (e.g. for the same subject, or fordifferent subjects), and storing, in a database, or suitable datastructure, the local tissue diagnostic data obtained from local tissuediagnostic measurements, and tissue analysis data.

For example, one entry in a tissue analysis database may be constructedas follows, in which multiple diagnostic modalities are employed tointerrogate a local tissue region of a subject (although understood isthat an entry or data element may include diagnostic data from a singlediagnostic modality). In the present example, three different diagnosticmodalities are employed. A tracked Raman spectroscopy probe is employedto correlate the location the local tissue region with its Ramanspectrum. Intraoperative MRI is also be employed to obtain MRIdiagnostic data, where the use of a tracking and/or navigation systemwill allow the obtained MRI data to be correlated with the Ramanspectrum. Finally, a tracked optical imaging device is be employed tooptically interrogate the local tissue region, allowing the visualappearance of the tissue to be correlated with the Raman and MR data.

The local tissue diagnostic data associated with these measurements isstored, in the database (or other suitable data structure), along with,or correlated with, tissue analysis data pertaining to the local region.The tissue analysis data may include pathology data. For example, if atissue sample from the local tissue region is excised and sent to apathology laboratory, the pathology results (which may include celltype, microscope image, tissue imaging (for example, X-ray, MRI)) may becorrelated, as tissue analysis data, with the local tissue diagnosticdata that was intraoperatively obtained, and stored as an entry in thetissue database.

As described below, other types of tissue analysis data may additionallyor alternatively be correlated with the local tissue diagnostic data toform the entry (e.g. database element) of the tissue analysis database.Examples of other tissue analysis data include, but are not limited to,outcome data (e.g. pertaining to the outcome of a medical procedureduring which the local tissue diagnostic data was obtained), diagnosisdata (e.g. pertaining to the diagnosis of a given pathology), andadditional data pertaining to the subject (such as, but not limited to,demographic data, genetic data, and/or medical history data). In someexample embodiments, the diagnostic data from two or more differentdiagnostic modalities may be employed allow for improved tissueanalysis, as the same tissue region can be measured using multipletissue analyses.

The tissue analysis database, which, as noted above, includes tissueanalysis data from prior tissue analyses, may be used to guide, orsuggest, which diagnostic modalities should or could be employed whenperforming medical procedures (e.g. surgical tissue resectionprocedures) involving known types of tissue. For example, if tissueresection of a known tissue type (e.g. a known tumor type) is to beperformed, then the tissue analysis database can be searched to identityprior tissue analyses corresponding to the particular tissue type ofinterest, in order to identify diagnostic modalities that have beenshown to have local tissue diagnostic data that is correlated with agiven tissue type. The identified diagnostic modalities may then beemployed during the tissue resection procedure, and the local tissuediagnostic data that is intraoperatively obtained may be compared withthe archival local tissue diagnostic data to intraoperatively identifyexposed tissue. In such an embodiment, it may be beneficial to filterthe tissue identification database such that any local tissue diagnosticdata that is included in the tissue identification database exhibitsdiagnostic data that is correlated with the tissue analysis data (e.g.such that local tissue diagnostic data that did not show features or asignature associated with the tissue type is excluded).

In one example embodiment, a tissue resection procedure may be plannedfor a known, or suspected tissue type. FIG. 11 is a flow chartillustrating a method of selecting suitable diagnostic modalities foruse during a medical procedure, and using the suitable diagnosticmodalities to perform a similarity analysis between local diagnosticdata and archival local tissue diagnostic data stored in a tissueanalysis database. For example, a tissue resection procedure may beplanned for a clinical case in which the tissue type is known to be, orsuspected to be, a glioblastoma tumor. As shown at step 620, the tissueanalysis database then be searched for diagnostic data pertaining to theknown or suspected tissue type, in order to identify one or moresuitable diagnostic modalities at step 625.

In the present example of FIG. 11, the tissue identification databasewould be searched for database entries pertaining to glioblastomatumors, in order to identify diagnostic modalities associated with suchdatabase entries. For example, the search of the tissue analysisdatabase may identify suitable diagnostic modalities as Ramanspectroscopy, T2 MRI imaging, and ADC MRI imaging. The tissue analysisdatabase may include entries whereby glioblastoma tumors have beenassociated with: Raman spectra having a specific spectral signature, T2MRI image data in which the tissue appears dark, and bright ADC MRIdata.

Accordingly, based on the knowledge of these diagnostic modalities asbeing suitable for intraoperative diagnostic measurements of this tissuetype, a subsequent medical procedure involving tumor resection of aglioblastoma tumor (e.g. based on a suspected pathology, or based on apreviously performed biopsy) may be performed diagnostic devicesemploying these diagnostic modalities, as shown at step 630.

During the medical procedure, tumor tissue may be intraoperativelydetected by associating local tissue regions of dark T2 MRI, bright ADC,and also having the specific Raman spectra, with the glioblastoma tumor.Such an embodiment may enable greater and more precise margindelineation and simultaneously more complete tumor excision with morehealthy brain sparing.

Another example embodiment is shown in FIG. 12. FIG. 12 is a flow chartillustrating an example method of determining similarity among differenttissue regions based on spectroscopic measurements. During a medicalprocedure, a biopsy sample may be obtained, as shown at step 640. Thissample may be measured using a spectroscopic diagnostic modality, suchas a Raman spectroscopy, or hyperspectral imaging, in order to obtain acharacteristic spectral signature associated with the tissue. Forexample, it may be known that the tissue type is tumor tissue. Thisknowledge of the tissue being tumor tissue may result from obtaining atissue sample within the central region of a tumor. The spectroscopicmeasurement that is obtained from the tissue sample may be performedex-vivo, using a suitable ex-vivo imaging device.

The local tissue diagnostic data obtained from the diagnosticmeasurement of the biopsy sample, having a characteristic spectralsignature, may be subsequently compared with intraoperative measurementsfrom an intraoperative, in-vivo, spectroscopic diagnostic device (suchas a Raman probe, or a hyperspectral probe) in order to determinewhether or not a local tissue region (e.g. a layer of exposed tissueremaining after tumor resection) is of the same type as the tissuesample, as shown at step 650. This determination may be made byperforming an assessment of the similarity between the measured spectralsignature from the tissue sample, and the in-vivo spectrum (or spectra)intraoperatively obtained by measuring the tissue, as shown at step 655.Such an embodiment may be employed to obtain an indication of when atumor margin has been reached, and/or to confirm an alternative basisfor determining that a tumor margin has been reached (such as anestimate based on a surgical plan).

As noted above and described in the flow chart shown in FIG. 10, thetissue analysis database may be intraoperatively accessed and searchedduring a medical procedure in order to obtain tissue analysisinformation pertaining to unidentified tissue. During such a medicalprocedure, a local region may be measured in-vivo by performing one ormore local tissue diagnostic measurements. For example, when performinga medical procedure in which one or more local tissue diagnosticmeasurements are made on an unidentified tissue region, the local tissuediagnostic data obtained from such diagnostic measurements may becompared with the archival local tissue analysis data in the database,in order to search for a prior tissue analysis having similar localtissue diagnostic data.

In the event that a prior local tissue analysis is identified, thetissue analysis data associated with the similar prior local tissueanalysis may be provided and optionally processed such that it may beemployed for various purposes and applications. For example, inembodiments, in which the pathology of tissue is unknown, pathologyresults from one or more similar prior tissue analyses can be presentedwhen identified based on a search of the tissue database (e.g. as shownin FIG. 8A).

In order to avoid the system to be considered a fully automated computerbased diagnosis system, the system can present one or more similarconfirmed pathology cases that can be examined by the operator. Thisdata can be reviewed, and the associated imaging examined in visualcomparison.

Referring again to FIG. 8B, in some example embodiments, archived localtissue diagnostic data that is deemed to satisfy similarity criteria maybe presented to an operator using a graphical user interface. In oneexample implementation, in which the similar archived local tissuediagnostic data is imaging data, the region in the imaging set with theconfirmed pathology may be scaled and presented in a similar size,orientation, and/or configuration as the active case (the case which thephysician is comparing against), in a compare mode. Noted is that thismay not provide a definitive determination of tumor pathology based onimaging—and instead may provide an informative and suggestive search andretrieve functionality that presents the appropriate data based onsearch results.

In one example embodiment, the local tissue analysis database may alsoinclude time-dependent medical procedure data recorded during a medicalprocedure in which one or more prior local tissue analysis wereperformed. This may allow a surgeon or operator to replay one or moreportions of a similar prior tissue analysis identified according to theaforementioned methods. The time-dependent medical procedure data mayinclude time-dependent positional information associated with theposition and the orientation of one or more medical instruments employedduring the medical procedure, which may be processed to render anddisplay a time-dependent simulation of one or more of the medicalinstruments during at least a portion of the medical procedure.

In one example embodiment, the method may be adapted to provide acomputer recommended surgical plan, or an evaluation of a user selectedsurgical plan. For example, the tissue analysis database may includesurgical plan data for at least one similar prior local tissue analysis.This may enable an operator, having identified a similar tissue analysisaccording to one of the methods described above, to review, or simulate,one or more steps of the surgical plan that was employed for theprevious tissue analysis.

In another example implementation, a proposed surgical plan, outlining aplanned medical procedure for a subject, may be compared with surgicalplan data stored in the tissue analysis database. This may allow aclinician to observe or review outcomes of similar surgical plans storedin the database.

In some example embodiments, the method may be adapted to provide acomputer recommended surgical plan, or an evaluation of a user selectedsurgical plan. For example, the database may be searched for similarapproaches that were taken for similar tumors, tissues and criticaltissue structures. The search can be done in a similar manner asdescribed as above, but instead of searching for the optimal outcome,the search is performed to best match the user selected approach.

FIG. 13 provides a flow chart illustrating an example method employing atissue analysis database, where in the present example embodiment, thetissue analysis database includes to identify prior tissue analysishaving a similar surgical plan that was performed. At step 660,preoperative surgical plan data associated with medical procedure to beperformed on a subject is obtained. The tissue analysis database is thenaccessed at steps 662 and 664 to obtain outcome data and archivalsurgical plan data associated with one or more prior medical procedures.The Preoperative surgical plan data and archival surgical plan data arethe compared according to pre-selected similarity criteria at step 666,and one or more similar prior medical procedures having archivalsurgical plan data satisfying pre-selected similarity criteria areidentified at step 668. If one or more prior medical procedures areidentified having similar surgical plan data, outcome data associatedwith the similar medical procedures may be provided. The outcome datamay be used, for example, to infer to potential effectiveness of a givenchoice of surgical plan.

Methods of Performing Similarity Analysis with Prior Tissue Analyses

Many of the embodiments described in the present disclosure involve theassessment of similarity between local tissue diagnostic data pertainingto tissue analysis performed on a subject, and archival local tissuediagnostic data stored in a database, in order to identify one or moresimilar prior local tissue analyses. Performing such similarityassessment first requires the selection of appropriate metrics orcriteria that differentiate the data set in the clinical context. In thespecific case where local diagnostic measurement is composed of images,an approach in computer vision is to detect and describe local featuresin images. Such features are referred to as “keypoints.” A method suchas Scale-Invariant Feature Transform (SIFT, U.S. Pat. No. 6,711,293) isan example of algorithm that is used for the purpose of detecting anddescribing local features that aid in searching image data sets. Secondstep of the search involves judicial reduction of the size of the dataset based on additional context associated with the data set. First,selection and measurement of appropriate criteria is explained.

In some non-limiting examples, the clinical data set may include imagingcontrast dynamics (contrast in-flow, contrast out-flow), diffusioninformation (e.g. FA or ADC mapping), quantitative T1 and T2, CTcontrast flow, PET tracer dynamics. In all of these cases, the resultingdata is a set of images, the images can be decomposed into essentialcriteria using feature extraction algorithms such as SIFT, SURF (HerbertBay, Andreas Ess, Tinne Tuytelaars, Luc Van Gool, “SURF: Speeded UpRobust Features”, Computer Vision and Image Understanding (CVIU), Vol.110, No. 3, pp. 346-359, 2008) and Principal Component Analysis. Oncethe data is reduced to essential criteria, searching can be done in alower dimensional space that comprises the essential criteria instead ofthe entire image data points.

The process of matching a new clinical image data with an archival dataset is illustrated in FIG. 14A and FIG. 14B. In one non-limiting example(shown in FIG. 14A), the process starts with decomposition of the newlyacquired image data (step 680) into essential criteria (step 682) andcomparison of the newly estimated essential criteria values againstessential criteria values already stored in a database (step 684).)(e.g. calculating a vector distance). Examples of searching algorithmsinclude nearest neighbor classifier (step 686), Bayesian classifier andthe general class of clustering algorithms.

FIG. 14B illustrates another method for comparing patient-specific datawith those in the archival data set. Here, the criteria are estimatedfrom the data in the archival data set (step 690) and these criteria aregrouped into clusters (step 692) described by multi-dimensional Gaussiandistributions (reference: “Findings Groups in Data: An Introduction toCluster Analysis,” L. Kaufman and P. J. Rousseeuw, Wiley Series inProbability and Statistics). Preoperative and intraoperative data from aspecific patient are then obtained (step 694) and criteria are estimatedfor this data (step 696). Conditional probabilities are finallyestimated (step 698) to identify the most likely Gaussian distributionwhere the newly acquired data belongs. Clinical information (step 699),such as surgical outcomes, survival rate, postoperative complications,quality of life (QOL) measures, from archival data set can then be usedto anticipate and appropriately prepare for the care of the patient.Even though these tools are readily available, above described approachdoes not utilize clinically relevant criteria; instead, abstractcriteria are extracted and compared. This approach can be biased byspurious noise in the data and rationale for identifying a match cannotoften be easily discerned.

Additional criteria such as tumor size, tumor location in brain anatomytumor histology and pathology results may be used in combination withthe imaging results to add clinically relevant criteria to the dataset.Thus, the range of data set that is searched (also referred to as“search space”) is comprised of two sets: (i) essential criteriaobtained by decomposing images and (ii) above described additionalclinically relevant criteria. In one non-limiting example, tumor size isestimated by segmenting clinical image data (such as MR and CT) toisolate the tumor and then computing its surface area or volume.Segmentation of the image data to isolate the tumor may be done in 2Dspace in each of the original image slices that are used to constructthe 3D volume data (e.g. MRI 2D image slices). Segmentation of 2D imagemay be accomplished using region-based segmentation methods (e.g. regiongrowing, region splitting and merging, unsupervised segmentation), dataclustering methods (e.g. hierarchical clustering, partitionalclustering) or edge-based segmentation methods (e.g. watershedsegmentation, markers based approach). These methods are in the computervision industry and are explained in the context of generic image dataand clinical images in “Tutorial: Image Segmentation,” Yu-Hsiang Wang,Graduate Institute of Communication Engineering, National TaiwanUniversity, Taipei, Taiwan, ROC. Natively 3D segmentation approaches mayalso be used.

Tumor location relative to the brain anatomy may be deduced first basedon the identified location of the tumor relative to the brain. This maybe achieved by identifying the centroid of a tumor region describedpreviously. Then, this centroid location may be expressed relative tothe patient's brain. Finally, translating this location to a generalbrain atlas such as that described in The Cerefy Brain Atlases(“Continuous Enhancement of the Electronic Talairach-Tournoux BrainAtlast,” Neuroinformatics, Vol. 3, 2005) will enable the association ofanatomical location of the specific tumor. This identification oflocation can be achieved by applying standard image co-registrationtechniques between the patient imaging and atlas exemplars.

Tumor histology may be quantified using imaging techniques such asApparent Diffusion Coefficient (ADC) measured from diffusion weightedresonance (MR) imaging methodologies. Past publications have illustratedcorrelation between ADC and cellularity (reference: “Diffusion-weightedMR imaging of the brain: value of differentiating between extraaxialcysts and epidermoid tumors,” J S Tsuruda, W M Chew, M E Moseley and DNorman, American Journal of Roentgenology, Nov. 1990, Vol. 155, No. 5).Hence, the correlation can be used to build a mathematical model thattransforms measured ADC values to cellularity measures. In summary, allof the above clinically relevant criteria may be quantified andassociated with clinical images to arrive at a data set that issearchable based on clinically relevant criteria.

Finally, pathology information may be also added as a clinicallyrelevant search criteria as follows. Biopsy is often performed prior tosurgical resection of the tumor. The removed tissue sample may beanalyzed using biochemical processing or monoclonal anti-bodies at apathology laboratory to identify the type of tumor (reference:“Identification of proliferating cells in human gliomas using themonoclonal antibody Ki-67,” Zuber et. al., NeuroSurgery, 1988, Vol. 22,Issue 2). Similarly, some of the tissue samples resected during surgicalprocedures are set aside for such pathology analysis. The informationfrom such pathology analyses may be added to the data set collectedduring the surgical procedure by associating the results of pathologyanalysis with the exact location and time point where the analyzedtissue sample was removed. Thus, the pathology analysis results can beconsidered as metadata associated with the original position and timeinformation captured during the recording of surgical procedure.

The metadata may be searched independent of position and timeinformation to identify past surgical procedures (in the archival data)that are similar to a particular surgical procedure. In this case, thesearch algorithm may be a text-based searching algorithm that is used insearching text strings in large volumes of text data (reference: “A faststring searching algorithm,” Comm Association for Computing Machinery,20(10): 762-772). Identification of the pathology of the particularpatient with those in the archival data set may aid in the evaluation ofclinical outcome that may be anticipated for the particular patientbased on what was observed for patients with similar pathology in thepast.

A similar means of searching through established databases is possibleusing specific metrics from the local non-imaging diagnostic modalities.For example, metrics such as, but non-limited to, metaboliteconcentrations or ratios, local scattering or absorption, tissuestiffness (elasticity), anisotropy, etc., can be used.

In other cases, local imaging modalities may be employed which mayprovide enhanced imaging quality, resolution, and/or the ability tomeasure contrast mechanisms that cannot otherwise be imaged withexternal imaging probes. Imaging in this manner provides a uniqueopportunity to correlate quantitative metrics to the tissue sample ofinterest, and can facilitate diagnosis, registration between regionaland local imaging, and provide a means to track the biopsy samplethrough detection to diagnosis through pathologic examination.

Additional information that may be analyzed to assess similarity mayinclude specific patient information such as age, weight, the presenceof certain genetic mutations, exposure or presence of viruses, ordisease prevalence information. For example, in each of these morequalitative metrics an appropriate quantitative ranking may be providedand associated with the features to enable a similarity calculation.

A non-limiting example of associating scores with patient information isthe association of cancer recurrence score with tumor size and tumorgrade (reference: “A population-based study of tumor gene expression andrisk of breast cancer death among lymph node-negative patients,” Habelet. al., Breast Cancer Research 2006, Vol. 8, No. 3). Hence, previouslydescribed tumor size estimates may be associated with disease prevalenceinformation. The latter information is another metadata that can be usedas search criteria to identify other patients in the archival data set.Such identified information can be used to infer possible outcome of thecurrent surgical procedure, such as survival period and quality of lifeafter surgery. This information, in turn, can aid in choosing suitablepost-surgical care for the particular patient. For example, if the tumorsize indicates a high possibility (high score) for cancer recurrencethen the patient can be watched more carefully with frequent imaging tomonitor possible recurrence.

In some embodiments, the search of the tissue analysis database, and theassociated similarity analysis, can be modified to include weightingfactors, such that one or more criteria are assessed as having a greaterweighting in the similarity analysis. In one example implementation, oneor more weighting factors can be pre-selected. In another exampleimplementation, one or more weighting factors can be associated with adecision tree. In yet another example implementation, one or moreweighting factors may be selected by an operator (e.g. dynamicallyselected during a medical procedure). For instance, for a medicalprocedure involving a tumor that has been identified, or is suspected tobe, a Stage 4 Glioblastoma multiforme (GBM), the weighting of the MRIcontrast may be weighted as a higher number, for instance 2, verses allother factors weighted as one. As a contrasting example, in the case ofa Stage 2 GBM, the weighting of the DWI ADC value may be weightedhigher, for instance 3, verses all other factors weighted as one. Inanother embodiment, the features weighting can be determined based on aglobal calculation across multiple data points for a patient, or acrossa large patient population.

In one embodiment, a weighting can be applied based on the operator'sinterpretation of the results. For instance, if the surgeon determinesthe local Raman signal to be of high-quality, and clearly delineating adifference in tissue within that subject, they may choose to weight thatfactor higher than other factors such as CT perfusion of the area. Theymay also choose to exclude regions, or down-weight specificparameters—such as ADC values from a region of interest, if it appearsthe registration accuracy between the image sets is poor, or the imagingquality is not sufficient. In addition, local intraoperative imaging canbe registered back to the radiology presentation of the data. In thefuture this can better inform diagnosis as discussed in the context ofdiagnostic (radiologic) utilization of the system. In this way, higherquality imaging can be used as an internal surrogate to tissue samplingand pathology analysis, and may in-fact be more accurate than thecurrent accepted gold standard of tissue characterization.

EXAMPLES

The following examples illustrate non-limiting example implementationsof various aspects of the present disclosure, within the context ofpreoperative, intraoperative, and postoperative neurosurgical proceduresinvolving the resection of brain tumors. The examples presented hereinare provided to enable those skilled in the art to understand and topractice embodiments of the present disclosure. They should not beconsidered as a limitation on the scope of the disclosure, but merely asbeing illustrative and representative thereof.

Example 1 Preoperative Analysis and Search of Tissue Analysis Database

In a first example, shown in FIG. 15, various aspects of the presentdisclosure may be employed in the context of preoperative proceduresinvolving preoperative imaging and a tissue biopsy. FIG. 15 is a diagramillustrating the interrelationship between various aspects of an exampleimplementation of the methods described above, showing how variousdiagnostic measurements and inputs are employed in order present searchresults based on a search of a tissue identification database 500. Thepresent example implementation involves specific utilization of regionalimage data 700, local diagnostic measurements 702, and tissue analysispathology data 704 to link imaging and pathology results in a singlepatient, and linking results across subjects.

Shown in the middle of the figure, is an axial view of a brain 530, witha tumor 532 originating in the ventricle of the brain, and growing tothe surface of the brain. The tumor is shown as three differenttextures, representing three different types of tumor cells. This isrepresentative of tumors, which can be heterogeneous in their biology,and thus their appearance on imaging, pathology, and their response totreatments.

A tracked instrument 534 is shown relative to the tumor and pointing toa location in the tumor. When used in conjunction with a trackingsystem, and a tip tracking strategy (as is needed for flexibleinstruments, i.e. greater than 1 mm flex at the tip inside the tissue ofinterest, or the resolution of interest for that procedure), the exactposition of the surgical device is known relative to the tissue ofinterest with great certainty.

If this instrument 534 is used in conjunction with a biopsy samplingdevice, and the biopsy instrument actuation is measured in coordinationwith the sample removal, and the sample is stored or tracked in a mannerthat it can be uniquely identified relative to this location atpathologic analysis, then this pathology results can be recorded anddisplayed relative to the location for which the sample was retrieved.

As noted above, the location can represent on an image viewing system,identifying a reference marker (e.g. a label, or tag) on a regionalimage 700, that corresponds to the location at which the sample wasacquired. Upon selection of that label, the system may present thecorresponding pathology report on the screen for viewing. If there weremultiple locations from which multiple samples are selected, each wouldhave an associated report. This was explained previously in relation toFIG. 8A.

The navigation system would have associated tip position accuracy at theparticular time when the sample was taken, for instance due toregistration error and imaging related distortions, total accuracy forthe tip could be estimated, and displayed as a visual representationaround the center of that estimated tracked needle tip location (as acircle, or a sphere or like representation).

In one embodiment, if the user selected the point for which thepathology results were linked, a search of database 500 could beperformed to look for other studies, which had similar metrics. In somenon-limiting examples, the metrics 730 could include quantitativeimaging metrics such as imaging contrast dynamics (contrast in-flow,contrast out-flow), Diffusion information (ADC mapping), quantitative T1and T2, CT contrast flow, PET tracer dynamics. Additional metrics suchas tumor size, tumor morphology or histology, and pathology results maybe used in combination with the imaging results to characterize thelocation and enable a search. Even further metrics includes specificpatient information such as age, weight, or the presence of certaingenetic mutations. In each of these more qualitative features, anappropriate quantitative ranking must associate with the features.

As shown at 740, the database search can be weighted towards somemetrics 730 having a greater weighting in the search. The weightings canbe pre-set, or associated with a decision tree, or user selected. Forinstance, for a tumor that demonstrates a Stage 4 Glioblastomamultiforme (GBM), the weighting of the MRI contrast may be weighted as ahigher number, for instance 2, verses all other factors weighted as one.As a contrasting example, a Stage 2 GBM, the weighting of the DWI ADCvalue may be weighted higher, for instance 3, verses all other factorsweighted as one. In another embodiment, the features weighting can bedetermined based on a global calculation across multiple data points fora patient, or across a large patient population.

In the case of diagnostic imaging, where the pathology is unknown, a setof comparable pathology results can be presented based on a similarsearch method previously presented. In order to avoid the system to beconsidered a fully automated computer based diagnosis system, the systemcan present to the user a set of similar confirmed pathology cases thatcan be examined by the user, as shown at 742. These sets can bereviewed, and the associated imaging examined in visual comparison.

In order to make the viewing efficient, the region in the imaging setwith the confirmed pathology would be scaled and presented in a similarconfiguration as the active case (the case which the physician iscomparing against), in a compare mode. This is not a definitivedetermination of tumor pathology based on imaging—but a search andretrieve function that presents the appropriate data based on searchresults.

If local in-vivo imaging 702 is acquired at the same time that thebiopsy sample is acquired, then the in-vivo imaging can be presented inthe same context, and location as the tag for the pathology results, orif there are no corresponding pathology results, in place of thoseresults. Examples of local imaging modalities 702 include OCT,high-frequency ultrasound, Spectroscopy, MRI, MR Spectroscopy, tissueconductivity, electromagnetic imaging, etc.

A similar means of searching through established databases 740 ispossible using specific metrics from the local imaging. Metrics such asmetabolite concentrations or ratios, local scattering or absorption,tissue stiffness (elasticity), anisotropy, etc., can be used. In mostcases the local imaging will provide significantly enhanced imagingquality, resolution, or the ability to measure contrast mechanisms thatcannot otherwise be imaged with external imaging probes. Imaging in thismanner provides a unique opportunity to correlate quantitative metricsto the tissue sample of interest, and can facilitate diagnosis,registration between regional and local imaging, and provide a means totrack the biopsy sample through detection to diagnosis throughpathologic examination.

Shown on FIG. 15 are the various inputs (regional imaging 706, localimaging or other diagnostic measurements 702, and pathology data 704) toa software engine 740 that determines various metrics 730 (imagingmetrics, tumor morphology, tumor size/location, point imaging metrics,and pathology results). These metrics 730 can contribute to associateddatabase for that subject, as well as be used in a search of thespecific subject database, or other databases to locate similar imaging,pathology, or anticipated outcomes for that subject, as shown at 500.

In one example implementation, the search may be performed using asearch algorithms as previously described, with a weighting of thefeatures used determined by, for example—pre-set weightings,contextually selected weightings, user selected, or modified weightings,weightings selected or modified based on data quality measures (e.g. ifit appears that the fidelity of that measure is poor, it is rejected),and weightings selected in an adaptive, or trained manner

The results may be presented, as shown at 742, in a manner that ispre-selected, contextually, or user selected, for instance alwayspresented as a top ten ranking of subjects from the local institutiondatabase. It may be desired that these large datasets be accessible in afast manner, therefore pre-loading of the typically viewed number ofsets could be accomplished during processing of the search results.

Example 2 Preoperative Analysis and Search of Tissue Analysis Database

Another example implementation demonstrating of the clinical utility ofthe system is the use of an example embodiment of the system in contextsurgical and treatment planning for the subject, as illustrated in FIG.16. FIG. 16 is a diagram showing an example embodiment involvingspecific utilization of preoperative imaging, pathology, and pointsource imaging data to facilitate decision making for treatment andsurgical planning Data from the results of previous surgical treatmentsperformed on that subject, or subjects with similar imaging and/orpathology are presented based on an algorithmic search.

In this case, based on preoperative imaging 706, pathology results 708,and knowledge of the devices or system used in treatment (i.e. devicephysical properties, treatment modality properties), an expected outcomefor the subject may be suggested based on subjects with similar imaging,pathology and medical condition. The optimal approach may be selectedfor the subject while using this information in the context of treatmentor surgical planning

Shown in the center of the figure, is an axial view of a same brain 530as in FIG. 15; however, this time with a surgical implement 772represented in the same coordinate frame. In this case, the implement isillustrated as a surgical access port, with an introducer device.

In the present example embodiment, the model of the associated device,structures and tumor from the preoperative images, can then be used, forexample, to provide a computer recommended treatment plan, or anevaluation of a user selected treatment plan.

For the selection of a computer recommended treatment plan, the softwaresystem can search (as shown at 744) the database 500 for tumors thathaving various metrics or criteria 732, such as tumors of similarpathology, similar size, and located in a similar location in the brain,with corresponding nerve fascicle and critical structure locationsundergone a similar surgical approach, i.e. with a similar device (forexample access port). Corresponding nerve fascicles can be identified byassociating tumor location with a brain atlas. Further, DTI atlas (suchas that available at www.dtiatlas.org) may be used to identify nervefascicles in the region.

Various properties associated with a tissue region of interest may beobtained from preoperative imaging 706. For instance, a tumor volume maybe determined from the images from multiple modalities. Tumor metrics732 such as stiffness can be determined or inferred using the externalimaging (DWI, T2, MR Elastography), internal imaging if performed inconjunction with biopsy (OCT measured stiffness, tissue anisotropy,biosensors, i.e. direct pressure measurement), or pathology wouldcontribute to the model of the tumor (vascularity, tissue fibrosis).Finally, preoperative diagnostic information may be used to rank thevarious search criteria. For example, if the patient has been alreadydiagnosed with Glioblastoma, the search criteria may add higher weightto past surgical data in the archival data set or database 500 thatcorrespond to tumors located in the frontal and temporal lobes sinceGlioblastoma are typically confined to these regions.

The end result 746 could be a ranked set of cases with a selectedsurgical approach, the associated patient outcome and postoperativeimaging set. The patient outcomes could be quantified and an optimalvalue would be calculated based on weighted rankings

In an additional example embodiment, if the positions of the tools aretracked for those procedures, the surgeon can watch the position of thedevices, and the tools that were selected for that case. In this way,the surgeon can be informed of similar cases, their approaches, andtheir outcomes, in a way that can inform for their approach.

In a second method of using the model, the surgeon may select a surgicalpath for resecting the selected tumor, with or without using the modelin a predictive manner that searches a database, where the model couldinform upon the expected outcome.

In one non-limiting example of using surgical path as a search criteriathe path of the tools and the volume of tumor predicted to be surgicallyresected, can be used to search a database for similar approaches thatwere taken for similar tumors, tissues and critical tissue structures.The search can be done in a similar manner as described as above, andthe search is performed to best match the user selected approach.

For example, the search, in this case will may consist of first matchingthe planned surgical path to previously stored surgical paths in thearchival data set. The surgical paths can be represented as a set ofvectors that sequentially represent the entire path with each vectorrepresenting one segment of a piecewise surgical path. Congruencebetween thus described directed vector path and those previously storedin the archival data set can be estimated using such mathematicaltechniques as Hamming distance or Levenshtein distance (reference:“Error detecting and error correcting code,” Bell System TechnicalJournal, 1950, 29(2):147-160). These mathematical constructs provide ameasure of similarity between pairs of directed vector paths and, hence,can be used as a search criteria. The search can be further constrainedto those cases where estimated volume of the planned surgical procedurematches those in the archival data set. Hence, previous surgicalprocedures that closely match surgical path and tumor size can bepresented to the surgeon to review prior to embarking on the actualsurgical procedure.

The system may present a set of procedures 746 that had a particularsurgical approach in that region, that the user may be informed byviewing the imaging, and actual case (a recorded movement of thesurgical tools and imaging used in that region). In a similar manner,where surgical outcomes have been searched to mean patient outcomes, theeconomic impact of surgical approaches many be considered.

In one non-limiting example, the exact surgical tools used duringvarious stages of a surgical procedure may be recorded since eachsurgical tool may be uniquely identified and tracked by a navigationsystem. Each surgical tool, in addition, may have additional parameterssuch as capital cost of the tool, cost associated with each use (e.g.cost of disposable or consumable components associated with the tool)and cost associated with technical staff using the tool. In this manner,the total cost of using all the tools or specific tools in a surgicalprocedure may be computed and stored along with position and timeinformation of all the tools. This information may aid the surgical teamand hospital administration to accurately track the cost of varioussurgical procedures.

Example 3 Intraoperative Analysis and Search of Tissue Analysis Database

FIG. 17 illustrates an example embodiment involving specific utilizationof preoperative imaging (in the context of the surgical plan 712),intraoperative imaging 710 (full volume, regional, and point based),intraoperative pathology 716, and intraoperative procedure testing 714(electrophysiology for example), to facilitate decision making fortissue differentiation and treatment. Database 500 is searched based onmetrics 734, such that the results of previous surgeries performed onthe subject, or subjects with similar imaging and/or pathology arepresented 750 based on an algorithmic search 748. FIG. 17 illustrates anexample implementation, in which an embodiment of the system isconsidered in the context of surgical or treatment guidance. Shown inthe middle of the figure, in the center of the image, is an axial viewof a brain 530, with the overlay of an access port 782, and a trackedsurgical tool 784 (biopsy device, surgical resection device or pointimaging probe).

In FIG. 17, much of the tumor that was originally shown in FIG. 15 isremoved, and a small margin of tumor remains. It is in this context thatthe described system is most impactful. One of the biggest challenges insurgery is careful resection of the tumor at the boundaries of thetumor. By tracking the surgical tools, and registering imaging andpathology in the manner presented, the full informative and predictivepower of all associated imaging and pathology results can be used tobetter differentiate between tumor locations within various regions of asubjects tumor and brain.

In FIG. 17, much of the tumor that was originally shown in FIG. 15 isremoved, and a small margin of tumor remains. This is always a challengein tumor resection, balancing the goals of complete resection andminimizing damage to healthy tissue. By tracking the surgical approach,tools used, pathology, and any other relevant criteria, the system canoffer correlated retrospective information on resection performance(e.g. as narrowly as for a particular surgical team or procedure, orbroadly through all available records) to aid in informing surgicaldecisions. For instance, for a particular tumor size, tumor location,nerve fiber locations, pathology, patient specific information (age,virus status, genetic background), there would be an associated set ofprior cases with similar metrics. From these cases there would beinstances where margin of tumor was left behind from taking tooconservative of a tumor margin resulting in tumor being left behind, andvisible (for instance under MRI). Whereas other case where there was tooaggressive of a margin causing cognitive loss. In each casepre-operative imaging, such as MRI tumor volume as related to surgicalresection margin by way of overlapping volumetric measurement, orintraoperative imaging results such as Raman Spectroscopic signature ofthe tissue status at the margin, could be related to the boundary.Additional information such as the proximity of the margin to majorfiber tracts could also be measured relative to the margin to determinewhat proximity may lead to being too close causing nerve damage. Thisinformation would be presented to show the surgeon so they can use thisdata to guide the margin to fall between too aggressive, andinsufficient surgical margin resection, as guided by prior casesimilarity search in the manner presented. In a similar manner, the useof pressure sensors along surgical instrumentation, such as the edge ofthe port, or the edge of surgical retractors, could provide an average,or peak measurement of pressure on the surrounding tissue. A search ofclinical outcomes associated with similar instances of port positioning,or tissue retraction extent and location, would guide thresholds foracceptable peak pressure for that specific surgical procedure.

FIGS. 18A and 18B illustrate an example implementation in the context ofa view looking down the access port. Shown in this figure on the topleft is a view 800 of multiple types of tissues as seen by a video scanfrom a device such as an exo-scope. Here various colors, and texturescreate the appearance of multiple tissue types. Shown are five separateislands of tissue, with differing appearance, and lines that mayrepresent planes between tissues, vessels or nerve fibers. With only theoptical view of the tissue, it is not possible to differentiate betweenthe tissue types with certainty.

In FIG. 18C, a probe 810 is shown interrogating an island of tissue 815through access port 782. In one example implementation, this probe maybe a Raman probe. The associated tissue spectra obtained from the Ramanprobe is represented in the frequency spectra shown at 850 in FIG. 19,under the title “In-vivo Imaging Data.” This specific location in theport, and on the video image, can be located by tracking the position ofthe tip of the Raman imaging probe 810. This specific location may have,associated with it, additional local imaging data, either collected inthat surgical procedure, or from prior diagnostic imaging exams (thisinformation would be could registered through various registrationmethods). This specific location may also have associated regional, orvolumetric imaging data, for example, which may have been obtained inthis surgical procedure, or in prior pre-surgical scans.

If spatial registration can be adequately performed, then additionalmetrics associated with the other imaging data, such as contrast uptake,DWI ADC values, or other metrics useful in helping to differentiate orcharacterize the tissue, can be associated with this particular sampledpoint (or region outlined by a probe). In one non-limiting example, areformatted image, such as FIG. 18B, may be created by specificallyhighlighting all regions of the DWI ADC image that correspond to theparticular ADC value measured at the same location where the Raman datawas acquired. Image highlighting described here may be realized throughselective coloring of DWI ADC image pixels with specific valuesdescribed above.

Referring now to FIG. 18B, a view 805 is shown down access port 782 fromthe video scan, providing the above described reformatted and registeredview of an image down the same port trajectory. Registration will berequired because the DWI ADC image needs to be transformed to match theview down the port. In this representation of the tissues of interest,it can be seen that the islands of tissue have a similar appearance andclassification.

FIG. 19 is a diagram demonstrating how multiple tissue metrics areutilized to characterize tissue of interest, and how that information isused to search a database for similar tissue types characterized on asimilar metrics. The ranked similar tissue types are then associatedwith their respective full-imaging set, pathology, and subjectinformation, which are presented to the physician. As shown in FIG. 19,these features can be used to help differentiate the tissue of interestthat is being interrogated. This is represented at 850 by the box withthe label “In-vivo Imaging Data.” The point of interest can beassociated with different metrics or criteria, such as i) localquantitative data (such as ratios between peaks on the Raman spectra, orabsolute point quantities such as tissue anisotropy, or tissue densitycan be determined based on ADC measurements, as described before), ii)local imaging combined feature set, (such as tissue morphology on alocal scale), iii) intraoperative combined feature set (such as DWI,contrast flow, etc.), iv) preoperative imaging combined feature set(registered data including DWI, contrast flow, etc.). These metrics canthen be used to search a database for similar tissue, based on asimilarity analysis or algorithm, shown as “comparison” 860 in thefigure. The actual mechanism for comparison may be as described in FIG.14A and 14B.

The database search could be performed in a similar manner of weighting,and ranking described previously. In some embodiments, the search may beperformed based on diagnostic data obtained through measurements ofother regions that were done within the same subject. In the context ofimaging through a small surgical window to address a larger area ofinterest surgically, the ability to piece together a full view of theregions of interest can be lost as the surgeon traverses through areasof tissue.

As specific regions are interrogated in above described manner againstpreviously stored data in archival data set, some of the interrogatedregions may be stored for easy recall during the same procedure. Inother words, the database or archival data set that is searched is nowcomposed of the original archival data set and any new tissues regionsthat were interrogated recently. The same search algorithm may beemployed to search any new data against this expanded data set. Hence,the surgeon may explore the possibility of comparing a new tissueregions against previously analyzed tissue region. For example, theRaman spectroscopy data from a new tissue region may be compared withRaman spectroscopy data from a previously analyzed tissue region in thesame patient.

The surgeon may use this information simply to inquire whether thetissue under local interrogation is simply similar to tissue adjacent ormore similar to tissue that was seen previously, and clearlydifferentiable based on other imaging metrics. Accordingly, the surgeonis provided with the ability to reference information that waspreviously available, and interrogate tissue using various tissuemetrics.

In a further example of what has been described, intraoperative imagingand preoperative imaging can be combined to better define set ofpathology types based on similarity metrics.

Example 4 Intraoperative Analysis and Search of Tissue Analysis Database

FIG. 20 illustrates another example embodiment involving specificutilization of postoperative imaging 720, postoperative pathology data724, and postoperative therapy outcomes 722, to facilitate decisionmaking for next stage care, in the context of expected outcomes. Asshown in FIG. 20, multiple tissue metrics may be utilized tocharacterize tissue of interest, and how that information is used tosearch a database for similar tissue types characterized on a similarmetrics database 500 is searched 752 using one or more metrics 736, suchthat the results of previous surgeries performed on the subject, orsubjects with similar imaging and/or pathology and or treatment response754 are presented based on an algorithmic search 752.

Shown centrally in the figure, is an axial view of a brain 530, with theoverlay of the residual tumor 792, and the nerve fascicles. In thisexample context, the tumor, and brain tissue is evaluated after thesurgical procedure either using a full-volume scan, or a regional scan.The evaluation may be done immediately after the surgery, or done atvarious intervals at any times after the surgery is done, and may beinformed based on patient response, or metrics from the databasediscussed in this disclosure (for instance a specific follow-up imagingregiment may be suggested for a certain tumor size, location and subjectage). The postoperative imaging 720 is used to evaluate the tumor, andtissue metrics such as tumor volume, total fiber volume locally,regionally and throughout the brain. In addition postoperative pathologydata 724 may be obtained, and registered to the imaging set.

Further still, post-surgery outcomes 722 may be used to definequantifiable metrics such as neurological deficits (motor response,visual field coverage, cognitive testing results, etc.). Each of thesemetrics can be calculated in part, or whole by the software system, andcan be compared to a database in manners previously discussed. Forinstance, a specific residual tumor volume, with specific imagingmetrics (DWI ADC, Contrast uptake), located in a specific region of thebrain, and subject age, may be searched against similar metrics in thedata base, or find the best long-term outcomes. The search, andpresentation of the data would be done in a manner similar to thatpresented in the second context of use—surgical planning It isessentially the same process, as the next clinical decision for thetreatment of the subject is based on a similar process of searching forsimilar conditions, and considering those outcomes in comparison to thecurrent subject, and specific context of their disease and subjecthistory thus far.

Another unique feature of the present disclosure may involve theknowledge that tumors are inherently heterogeneous, and that tumorbiology has a fundamental impact on treatment—therefore point-wisepathological analysis should be linked to regional treatment selections.The pathology linked to a specific region of interest can instructsecondary local imaging (in same procedure), treatments (in sameprocedure) or secondary imaging (secondary procedure), or treatments(secondary procedure). Specific bio-markers can be selected for, forwhich specific imaging targets may be selected, or specific therapies.These bio-markers may instruct selection from a set of “off-the-shelf”therapies or contrast agents, or specific targets may be developed forthat subject/tumor combination. Therefore, the response of theparticular subject to the therapy (surgical, radiation, chemotherapy),can guide further therapies in the context of the imaging, pathology,and a larger subject outcome database.

In summary, according to various embodiments of the present disclosure,tracking of preoperative imaging, intraoperative imaging, and pathologyresults in a system that can link the information back to the originaldiagnosis, and allow the information to be accessible in the operatingroom to be integrated between pathology, surgery and radiology. In thisway the actual results can be used to support better-informed diagnosticdecision making in the future. There does not exist an integrated systemin this manner today, as typically radiologists are separated from theactual outcomes and any intraoperative imaging that can be used tosupport pathology, or radiology is not integrated in any mannerLikewise, in the pathology lab, the imaging information can be providedto with respect to the location of interest from which the sample isobtained, and any imaging information associated with it. Pathologydiagnosis can be made more accurate if the context in which the sampleis obtained is provided, as well as protecting against any potentialprocessing errors (for instance a completely wrong imaging to pathologycorrelation may indicate the wrong sample, or a sample taken from thewrong area of interest).

As described below, many of the shortcomings associated with existingsolutions can be addressed through deeper integration of imaging andtissue biopsy. With this more integrated combination of regional andlocalized imaging, biopsy of samples, and localized, personalizedtreatment, the following problems with existing solutions may beaddressed in the present disclosure: the inability of existing solutionsto track locations of individual biopsy samples, and in-vivo imaging ofthe regions where those samples were selected from; the lack of anexisting method to locate virtual “sticky” points or “geo locations” inthe subject in-vivo that correspond to biopsy sample locations, criticalstructures, or tumor margin locations; the lack of an existing method tolink together preoperative, intraoperative, biopsy locations that arecommon; the lack of an existing method to perform intraoperative imagingof tissue using the same modality that was used to diagnose the tissue(for instance an MRI DWI system that performed imaging on the excisedsurgical sample); the lack of an existing method to locate small regionsof interest in the intraoperative system for local sampling or imaging;the lack of an existing method to search through databases ofinformation linking common preoperative imaging, intraoperative imaging,and pathology reports to better inform decision making at all levels;the lack of an existing method to link from radiology to locationspecific intraoperative imaging information (in this way, commonpreoperative imaging information can be linked to a set ofintraoperative imaging sets that are representative of thatinformation); the inability for a radiologist to link location specificpathology information back to imaging in a way that can better informcurrent and future clinical diagnosis and conversely; the inability forthe pathologist to access prior, or intraoperative imaging to betterinform decision-making; the inability, based on existing solutions, touse common imaging taken in-vivo and ex-vivo to ensure the pathologyspecimen is properly tracked through the clinical chain; the inability,based on existing solutions, to use the biopsy specific information,such as antibody status, or genetic status, to link to betterintraoperative imaging or therapy options either within the sameprocedure, or done on another occasion; and the inability, based onexisting solutions, to build up such a comprehensive database ofpathology, in-vivo and ex-vivo (preoperative, intraoperative andpostoperative), and link it in a system that can be used for a subjectat either the preoperative, the planning, surgery or treatment.

The specific embodiments described above have been shown by way ofexample, and understood is that these embodiments may be susceptible tovarious modifications and alternative forms. Further understood is thatthe claims are not intended to be limited to the particular formsdisclosed, but rather to cover all modifications, equivalents, andalternatives falling within the spirit and scope of this disclosure.

What is claimed:
 1. A method of performing intraoperative biopsytracking during at least one medical procedure, by way of an automaticsystem comprising: a control and processing system comprising aprocessor, a tracking system, and a tracking engine interfaced with theprocessor and the tracking system; a storage device interfaced with thecontrol and a processing system; and an external user input and outputdevice interfaced with the control and a processing system comprisingproviding a display device, the method comprising: comparing, accordingto pre-selected similarity criteria, biopsy analysis data and archivalbiopsy analysis data associated with at least one medical procedure byusing the processor, comparing comprising: detecting and identifying aplurality of local features in images relating to the biopsy analysisdata and images relating to the archival biopsy data by using computervision; detecting and identifying a plurality of local features in aninitial medical image by using a scale-invariant feature transformalgorithm to facilitate searching biopsy analysis data and archivalbiopsy data, using the scale-invariant feature transform algorithmcomprising: producing a difference image from the initial medical imageby blurring the initial medical image, thereby producing a blurredimage; subtracting the blurred image from the initial medical image,thereby producing the difference image; locating pixel amplitude extremain the difference image; defining a corresponding pixel region abouteach pixel amplitude extremum of the pixel amplitude extrema; producinga plurality of component subregion descriptors for each subregion of apixel region about the pixel amplitude extrema in the difference image;correlating the plurality of component subregion descriptors for eachsubregion of the pixel region about the pixel amplitude extrema in thedifference image with a plurality of component subregion descriptors foreach subregion of a pixel region about the pixel amplitude extrema inthe initial medical image; and indicating detection of each localfeature of the plurality of local features if a number of componentsubregion descriptors of the plurality of component subregiondescriptors defines an aggregate correlation exceeding a thresholdcorrelation; reducing a size of the biopsy analysis data and thearchival biopsy data by decomposing images relating to a clinical dataset using at least one feature extraction algorithm, the clinical dataset comprising data relating to at least one of: imaging contrastdynamics, diffusion information, quantitative T1 and T2, CT contrastflow, and PET tracer dynamics, whereby searching the biopsy analysisdata and the archival biopsy data is performed in a dimensional spaceless than that corresponding to all of the biopsy analysis data and thearchival biopsy data; and correlating apparent diffusion coefficient(ADC) with cellularity to build a mathematical model by using additionalclinically relevant criteria comprising at least one of: a tumor size, atumor location, a tumor histology, and a pathology result, therebytransforming measured ADC values into cellularity measures.
 2. Themethod according to claim 1, wherein comparing further comprises:obtaining and automatically recording the initial medical image of atleast a portion of a subject by using the tracking system; registeringthe initial medical image to an intraoperative reference frameassociated with the subject by the processor; intraoperatively trackingand automatically recording at least one biopsy instrument by using thetracking system; intraoperatively detecting and automatically recordingat least one location associated with at least one biopsyintraoperatively performed on the subject by using the tracking system;displaying at least one hyperlinked reference marker associated with theat least one biopsy on the medical image by using the display device,displaying comprising displaying the at least one hyperlinked referencemarker in relation to the at least one location corresponding to the atleast one biopsy, and displaying comprising displaying the at least onehyperlinked reference marker as at least one of a label and text,whereby a given hyperlinked reference marker is selectable from the atleast one hyperlinked reference marker, and whereby at least one ofassociated diagnostic data and biopsy analysis data is viewable;obtaining and automatically recording the biopsy analysis datacorresponding the at least one biopsy by using the processor; receivinginput, identifying a selected hyperlinked reference marker associatedwith a selected biopsy, by using the display device; and presenting atleast a portion of the biopsy analysis data associated with the selectedhyperlinked reference marker, by using the display device.
 3. The methodaccording to claim 2, further comprising: intraoperatively acquiringlocal in-vivo diagnostic data from a local diagnostic measurementperformed in a region associated with the at least one biopsy; and inthe event that local in-vivo diagnostic data is available for the biopsyassociated with the selected reference marker, presenting at least aportion of the local intraoperative diagnostic data associated with theselected reference marker, wherein displaying further comprisesdisplaying the at least one hyperlinked reference marker as at least oneof an icon and a tag, wherein comparing, according to the pre-selectedsimilarity criteria, comprises evaluating at least metric associatedwith at least one local tissue analysis performed on the subject and atleast one prior local tissue analysis, the at least one prior localtissue analysis associated with at least one of a medical history of thesubject and a plurality of medical histories associated with acollection of other subjects, wherein the biopsy analysis data isobtained intraoperatively, wherein receiving the input comprisesintraoperatively receiving the input, and wherein presenting the biopsyanalysis data comprises intraoperatively presenting the biopsy analysisdata.
 4. The method according to claim 3, wherein a diagnostic modality,associated with the local diagnostic measurement, comprises an imagingmodality.
 5. The method according to claim 3, wherein a diagnosticmodality, associated with the local diagnostic measurement, comprises anoptical modality.
 6. The method according to claim 5, wherein theoptical modality, associated with the local diagnostic measurement,comprises an optical imaging modality.
 7. The method according to claim5, wherein the optical modality, associated with the local diagnosticmeasurement, comprises Raman spectroscopy.
 8. The method according toclaim 5, wherein the optical modality, associated with the localdiagnostic measurement, comprises hyperspectral analysis.
 9. The methodaccording to claim 2, further comprising providing a visualrepresentation of an estimated error, associated with the accuracy ofthe at least one location corresponding to the at least one biopsy byusing the display device.
 10. The method according to claim 2, whereinthe biopsy analysis data is overlaid on the medical image.
 11. Themethod according to claim 2, wherein the biopsy analysis data isdisplayed in a separate window of a user interface.
 12. The methodaccording to claim 2, wherein obtaining and automatically recording themedical image is preoperatively performed.
 13. The method according toclaim 2, wherein obtaining and automatically recording the medical imageand intraoperatively performing the at least one biopsy areintraoperatively performed during the at least one medical procedure.14. An automatic system for intraoperative biopsy tracking during amedical procedure, the automatic system comprising: a tracking system;and a control and processing system comprising a processor and atracking engine interfaced with the processor and the tracking system, astorage device interfaced with the control and a processing system, thestorage device comprising a memory, and an external user input andoutput device, the external use input and output device comprising adisplay device, said memory storing instructions, which, when executedby said the processor, configures the control and processing system to:compare, according to pre-selected similarity criteria, biopsy analysisdata and archival biopsy analysis data associated with the at least onemedical procedure by using the processor to: detect and identify localfeatures in images relating to the biopsy analysis data and imagesrelating to the archival biopsy data by using computer vision; detectand identify local features in an initial medical image by using ascale-invariant feature transform algorithm to facilitate searching thebiopsy analysis data and the archival biopsy data, using thescale-invariant feature transform algorithm comprising: producing adifference image from the initial medical image by blurring the initialmedical image, thereby producing a blurred image; subtracting theblurred image from the initial medical image, thereby producing thedifference image; locating pixel amplitude extrema in the differenceimage; defining a corresponding pixel region about each pixel amplitudeextremum of the pixel amplitude extrema; producing a plurality ofcomponent subregion descriptors for each subregion of a pixel regionabout the pixel amplitude extrema in the difference image; correlatingthe plurality of component subregion descriptors for each subregion ofthe pixel region about the pixel amplitude extrema in the differenceimage with a plurality of component subregion descriptors for eachsubregion of a pixel region about the pixel amplitude extrema in theinitial medical image; and indicating detection of each local feature ofthe plurality of local features if a number of component subregiondescriptors of the plurality of component subregion descriptors definesan aggregate correlation exceeding a threshold correlation; reduce asize of the biopsy analysis data and the archival biopsy data bydecomposing images relating to a clinical data set by using at least onefeature extraction algorithm, the clinical data set comprising datarelating to at least one of: imaging contrast dynamics, diffusioninformation, quantitative T1 and T2, CT contrast flow, and PET tracerdynamics, whereby searching the biopsy analysis data and the archivalbiopsy data is performed in a dimensional space less than thatcorresponding to all of the biopsy analysis data and the archival biopsydata; and build a mathematical model by: using additional clinicallyrelevant criteria, comprising at least one of: a tumor size, a tumorlocation, a tumor histology, and a pathology result, and correlatingapparent diffusion coefficient (ADC) with cellularity, therebytransforming measured ADC values into cellularity measures.
 15. Thesystem according to claim 14, wherein the processor is furtherconfigured to: obtain and automatically record an initial medical imageof at least a portion of a subject by using the tracking system;register the medical image to an intraoperative reference frameassociated with the subject by using the processor; intraoperativelytrack at least one biopsy instrument by using the tracking system;intraoperatively track and automatically record the at least one biopsyinstrument by using the tracking system; intraoperatively detect andautomatically record at least one location associated with at least onebiopsy intraoperatively performed on the subject by using the trackingsystem; display at least one hyperlinked reference marker associatedwith the at least one biopsy on the medical image by using the displaydevice, the at least one hyperlinked reference marker displayed inrelation to the at least one location corresponding to the at least onebiopsy, and the at least one hyperlinked reference marker comprising atleast one of a label and text, whereby a given hyperlinked referencemarker is selectable from the at least one hyperlinked reference marker,and whereby at least one of associated diagnostic data and biopsyanalysis data is viewable; obtain and automatically record biopsyanalysis data corresponding to the at least one biopsy by using theprocessor; receive input identifying a selected hyperlinked referencemarker associated with a selected biopsy by using the display device;and present at least a portion of the biopsy analysis data associatedwith the selected hyperlinked reference marker by using the displaydevice.
 16. The system according to claim 15, further comprising adiagnostic measurement device operably coupled with the control andprocessing system, wherein said control and processing system is furtherconfigured to: intraoperatively acquire local in-vivo diagnostic datafrom a local diagnostic measurement performed in a region associatedwith the at least one biopsy by using the diagnostic measurement device;and in the event that local in-vivo diagnostic data is available for thebiopsy associated with the selected reference marker, presenting atleast a portion of the local in-vivo diagnostic data associated with theselected reference marker by using the display device, wherein the atleast one hyperlinked reference marker further comprises at least one ofan icon and a tag, wherein a comparison, according to the pre-selectedsimilarity criteria, comprises an evaluation of at least metricassociated with at least one local tissue analysis performed on thesubject and at least one prior local tissue analysis, the at least oneprior local tissue analysis associated with at least one of a medicalhistory of the subject and a plurality of medical histories associatedwith a collection of other subjects, wherein the biopsy analysis data isintraoperatively obtained, wherein the input is intraoperativelyreceived, and wherein the biopsy analysis data is intraoperativelypresented.
 17. The system according to claim 16, wherein a diagnosticmodality, associated with the local diagnostic measurement, comprises animaging modality.
 18. The system according to claim 17, wherein theimaging modality, associated with the local diagnostic measurement,comprises an optical imaging modality.
 19. The system according to claim16, wherein a diagnostic modality, associated with the local diagnosticmeasurement, comprises Raman spectroscopy.
 20. The system according toclaim 15, wherein said control and processing system is furtherconfigured to provide, in the displayed medical image, a visualrepresentation of an estimated error associated with the accuracy of thelocation corresponding to the at least one biopsy by using the displaydevice.